Browsing by Type "Other"
Now showing 1 - 6 of 6
- Results Per Page
- Sort Options
Item Open Access Actor prioritized experience replay (abstract reprint)(AAAI Press, 2024-03-24) Sağlam, Baturay; Mutlu, Furkan; Çiçek, Doğan; Kozat, SüleymanA widely-studied deep reinforcement learning (RL) technique known as Prioritized Experience Replay (PER) allows agents to learn from transitions sampled with non-uniform probability proportional to their temporal-difference (TD) error. Although it has been shown that PER is one of the most crucial components for the overall performance of deep RL methods in discrete action domains, many empirical studies indicate that it considerably underperforms off-policy actor-critic algorithms. We theoretically show that actor networks cannot be effectively trained with transitions that have large TD errors. As a result, the approximate policy gradient computed under the Q-network diverges from the actual gradient computed under the optimal Q-function. Motivated by this, we introduce a novel experience replay sampling framework for actor-critic methods, which also regards issues with stability and recent findings behind the poor empirical performance of PER. The introduced algorithm suggests a new branch of improvements to PER and schedules effective and efficient training for both actor and critic networks. An extensive set of experiments verifies our theoretical findings, showing that our method outperforms competing approaches and achieves state-of-the-art results over the standard off-policy actor-critic algorithms.Item Open Access Item Open Access Photocatalytic CO2 conversion: Beyond the earth(Elsevier, 2023-07-25) Low, J.; Zhang, C.; Karadaş, Ferdi.; Xiong, Y.The issue of climate change attributed to CO2 emissions has led to increased attention towards the study and development of artificial photosynthesis through photocatalytic CO2 conversion to recon‐struct the broken carbon cycle in nature. Photocatalytic CO2 conversion can simultaneously reduce the CO2 concentration in the atmosphere and produce valuable hydrocarbon fuels. With the recent discovery of abundant reserves of CO2 and water at extraterrestrial sites, it has been proposed that photocatalytic CO2 conversion can also be implemented at extraterrestrial sites to build up an artificial carbon cycle for providing propellants and life support for space missions. This comment presents our perspectives on the development of photocatalytic CO2 conversion beyond Earth, with a focus on its general principles and potential challenges that may arise at extraterrestrial sites. Finally, a brief overview of the future research directions in this field is presented.Item Open Access Rotation of microtubule asters under confinement(American Society for Cell Biology, 2023-02-01) Sami, A.; Stein, D. B.; Kabacaoğlu, Gokberk; Oakey, John S.; Sulerud, T.; Shelley, Michael J.; Gatlin, J. C.Item Open Access Self-organization of microtubules through hydrodynamic interactions drives cell-spanning rotational flows(American Society for Cell Biology, 2023-10-18) Stein, D. B.; Sami, A. B.; Kabacaoglu, GokberkItem Restricted Whiplash(2013-09-10) Chazelle, Damien, 1985-