Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Page Not Found

Page not found. Your pixels are in another canvas.

Jupyter notebook markdown generator

Posts

Not all jet-lags are the same

less than 1 minute read

Published: July 08, 2021

I wrote a blog post published on the networkpages on jet-lag. The post can be found at: https://www.networkpages.nl/not-all-jet-lags-are-the-same/

Synchronization in the body-clock

less than 1 minute read

Published: July 10, 2020

I wrote a blog post published on the networkpages on jet-lag. The post can be found at: https://www.networkpages.nl/synchronization-in-the-body-clock/

Synchronization

less than 1 minute read

Published: February 02, 2019

I contributed to a blog post on the networkpages. It can be found here: https://www.networkpages.nl/synchronization/

portfolio

Portfolio item number 1

Short description of portfolio item number 1

Portfolio item number 2

Short description of portfolio item number 2

publications

Large deviations for Markov processes with resetting

Published in Physical Review E, 2015

Markov processes restarted or reset at random times to a fixed state or region in space have been actively studied recently in connection with random searches, foraging, and population dynamics. Here we study the large deviations of time-additive functions or observables of Markov processes with resetting. By deriving a renewal formula linking generating functions with and without resetting, we are able to obtain the rate function of such observables, characterizing the likelihood of their fluctuations in the long-time limit. We consider as an illustration the large deviations of the area of the Ornstein-Uhlenbeck process with resetting. Other applications involving diffusions, random walks, and jump processes with resetting or catastrophes are discussed.

Recommended citation: Janusz M. Meylahn, Sanjib Sabhapandit, and Hugo Touchette. (2015). "Large deviations for Markov processes with resetting." Phys. Rev. E. 92, 062148. https://doi.org/10.1103/PhysRevE.92.062148

Synchronization of phase oscillators on the hierarchical lattice

Published in Journal of Statistical Physics, 2019

Synchronization of neurons forming a network with a hierarchical structure is essential for the brain to be able to function optimally. In this paper we study synchronization of phase oscillators on the most basic example of such a network, namely, the hierarchical lattice. Each site of the lattice carries an oscillator that is subject to noise. Pairs of oscillators interact with each other at a strength that depends on their hierarchical distance, modulated by a sequence of interaction parameters. We look at block averages of the oscillators on successive hierarchical scales, which we think of as block communities. In the limit as the number of oscillators per community tends to infinity, referred to as the hierarchical mean-field limit, we find a separation of time scales, i.e., each block community behaves like a single oscillator evolving on its own time scale. We argue that the evolution of the block communities is given by a renormalized mean-field noisy Kuramoto equation, with a synchronization level that depends on the hierarchical scale of the block community. We find three universality classes for the synchronization levels on successive hierarchical scales, characterized in terms of the sequence of interaction parameters. What makes our model specifically challenging is the non-linearity of the interaction between the oscillators. The main results of our paper therefore come in three parts: (I) a conjecture about the nature of the renormalisation transformation connecting successive hierarchical scales; (II) a truncation approximation that leads to a simplified renormalization transformation; (III) a rigorous analysis of the simplified renormalization transformation. We provide compelling arguments in support of (I) and (II), but a full verification remains an open problem.

Recommended citation: Garlaschelli, D., den Hollander, F., Meylahn, J. M., & Zeegers, B. (2019). "Synchronization of phase oscillators on the hierarchical lattice." Journal of Statistical Physics . 174(1), 188-218. https://doi.org/10.1007/s10955-018-2208-5

Properties of additive functionals of Brownian motion with resetting

Published in Journal of Physics A: Mathematical and Theoretical, 2019

We study the distribution of additive functionals of reset Brownian motion, a variation of normal Brownian motion in which the path is interrupted at a given rate and placed back to a given reset position. Our goal is two-fold:(1) for general functionals, we derive a large deviation principle in the presence of resetting and identify the large deviation rate function in terms of a variational formula involving large deviation rate functions without resetting.(2) For three examples of functionals (positive occupation time, area and absolute area), we investigate the effect of resetting by computing distributions and moments, using a formula that links the generating function with resetting to the generating function without resetting.

Recommended citation: Den Hollander, F., Majumdar, S. N., Meylahn, J. M., & Touchette, H. (2019). "Properties of additive functionals of Brownian motion with resetting." Journal of Physics A: Mathematical and Theoretical. 52(17), 175001. https://doi.org/10.1088/1751-8121/ab0efd

Two-Community Noisy Kuramoto Model Suggests Mechanism for Splitting in the Suprachiasmatic Nucleus

Published in Journal of Statistical Physics, 2020

Recent mathematical results for the noisy Kuramoto model on a 2-community network may explain some phenomena observed in the functioning of the suprachiasmatic nucleus (SCN). Specifically, these findings might explain the types of transitions to a state of the SCN in which 2 components are dissociated in phase, for example, in phase splitting. In contrast to previous studies, which required additional time-delayed coupling or large variation in the coupling strengths and other variations in the 2-community model to exhibit the phase-split state, this model requires only the 2-community structure of the SCN to be present. Our model shows that a change in the communication strengths within and between the communities due to external conditions, which changes the excitation-inhibition (E/I) balance of the SCN, may result in the SCN entering an unstable state. With this altered E/I balance, the SCN would try to find a new stable state, which might in some circumstances be the split state. This shows that the 2-community noisy Kuramoto model can help understand the mechanisms of the SCN and explain differences in behavior based on actual E/I balance.

Recommended citation: Rohling, J. H., & Meylahn, J. M. (2020). "Two-community noisy Kuramoto model suggests mechanism for splitting in the suprachiasmatic nucleus." Journal of biological rhythms . 35(2), 158-166. https://doi.org/10.1177/0748730419898314

Two-community noisy Kuramoto model

Published in Nonlinearity, 2020

We study the noisy Kuramoto model for two interacting communities of oscillators, where we allow the interaction in and between communities to be positive or negative (but not zero). We find that, in the thermodynamic limit where the size of the two communities tends to infinity, this model exhibits unstable non-symmetric synchronized solutions that bifurcate from the symmetric synchronized solution corresponding to the one-community noisy Kuramoto model, even in the case where the phase difference between the communities is zero and the interaction strengths are symmetric. The solutions are given by fixed points of a dynamical system. We find a critical condition for existence of a bifurcation line, as well as a pair of equations determining the bifurcation line as a function of the interaction strengths. Using the latter we are able to classify the types of solutions that are possible and thereby identify the phase diagram of the system. We also analyze properties of the bifurcation line in the phase diagram and its derivatives, calculate the asymptotics, and analyze the synchronization level on the bifurcation line. Part of the proofs are numerically assisted. Lastly, we present some simulations illustrating the stability of the various solutions as well as the possible transitions between these solutions.

Recommended citation: Meylahn, J. M. (2020). "Two-community noisy Kuramoto model." Nonlinearity. 33(4), 1847. https://doi.org/10.1088/1361-6544/ab6814

Two-community noisy Kuramoto model with general interaction strengths. I

Published in Chaos: An Interdisciplinary Journal of Nonlinear Science, 2021

We generalize the study of the noisy Kuramoto model, considered on a network of two interacting communities, to the case where the interaction strengths within and across communities are taken to be different in general. By developing a geometric interpretation of the self-consistency equations, we are able to separate the parameter space into ten regions in which we identify the maximum number of solutions in the steady state. Furthermore, we prove that in the steady state, only the angles 0 and π are possible between the average phases of the two communities and derive the solution boundary for the unsynchronized solution. Last, we identify the equivalence class relation in the parameter space corresponding to the symmetrically synchronized solution. The Kuramoto model is a model for studying the synchronization of oscillators (e.g., fireflies flashing). We consider two groups of oscillators, synchronizing within and across groups. Studying the stationary-states (or critical points) of the system leads to a system of equations that cannot be solved analytically. We introduce a geometric interpretation of these equations that allows us to analyze when and how many solutions are possible given a vector of model parameters. It also allows us to identify when symmetric solutions (solutions where the two groups are equally synchronized) and unsynchronized solutions occur.

Recommended citation: Achterhof, S., & Meylahn, J. M. (2021). "Two-community noisy Kuramoto model with general interaction strengths. I" Chaos: An Interdisciplinary Journal of Nonlinear Science . 31(3), 033115. https://doi.org/10.1063/5.0022624

Two-community noisy Kuramoto model with general interaction strengths. II

Published in Chaos: An Interdisciplinary Journal of Nonlinear Science, 2021

We generalize the study of the noisy Kuramoto model, considered on a network of two interacting communities, to the case where the interaction strengths within and across communities are taken to be different in general. Using a geometric interpretation of the self-consistency equations developed in Paper I of this series as well as perturbation arguments, we are able to identify all solution boundaries in the phase diagram. This allows us to completely classify the phase diagram in the four-dimensional parameter space and identify all possible bifurcation points. Furthermore, we analyze the asymptotic behavior of the solution boundaries. To illustrate these results and the rich behavior of the model, we present phase diagrams for selected regions of the parameter space. The two-community noisy Kuramoto model is used to study the synchronization on two communities of oscillators, interacting within and across the communities. The two-community structure is relevant for neurophysiologists (e.g., to describe the body clock13) and social scientists (e.g., to analyze polarized opinion formation5,12,18). The stationary states (or critical points) of the system solve a system of equations that cannot be solved analytically. We analyze where phase transitions occur, i.e., where new stationary states occur when varying the parameters of the model.

Recommended citation: Achterhof, S., & Meylahn, J. M. (2021). "Two-community noisy Kuramoto model with general interaction strengths. II" Chaos: An Interdisciplinary Journal of Nonlinear Science . 31(3), 033116. https://doi.org/10.1063/5.00226254

Learning to Collude in a Pricing Duopoly

Published in Manufacturing & Service Operations Management, 2022

Problem definition: This paper addresses the question whether or not self-learning algorithms can learn to collude instead of compete against each other, without violating existing competition law. Academic/practical relevance: This question is practically relevant (and hotly debated) for competition regulators, and academically relevant in the area of analysis of multi-agent data-driven algorithms. Methodology: We construct a price algorithm based on simultaneous-perturbation Kiefer–Wolfowitz recursions. We derive theoretical bounds on its limiting behavior of prices and revenues, in the case that both sellers in a duopoly independently use the algorithm, and in the case that one seller uses the algorithm and the other seller sets prices competitively. Results: We mathematically prove that, if implemented independently by two price-setting firms in a duopoly, prices will converge to those that maximize the firms’ joint revenue in case this is profitable for both firms, and to a competitive equilibrium otherwise. We prove this latter convergence result under the assumption that the firms use a misspecified monopolist demand model, thereby providing evidence for the so-called market-response hypothesis that both firms’ pricing as a monopolist may result in convergence to a competitive equilibrium. If the competitor is not willing to collaborate but prices according to a strategy from a certain class of strategies, we prove that the prices generated by our algorithm converge to a best-response to the competitor’s limit price. Managerial implications: Our algorithm can learn to collude under self-play while simultaneously learn to price competitively against a ‘regular’ competitor, in a setting where the price-demand relation is unknown and within the boundaries of competition law. This demonstrates that algorithmic collusion is a genuine threat in realistic market scenarios. Moreover, our work exemplifies how algorithms can be explicitly designed to learn to collude, and demonstrates that algorithmic collusion is facilitated (a) by the empirically observed practice of (explicitly or implicitly) sharing demand information, and (b) by allowing different firms in a market to use the same price algorithm. These are important and concrete insights for lawmakers and competition policy professionals struggling with how to respond to algorithmic collusion.

Recommended citation: Meylahn, J. M., & den Boer, A. V. (2021). "Learning to Collude in a Pricing Duopoly" Manufacturing & Service Operations Management . 0(0). https://doi.org/10.1287/msom.2021.1074

Limiting dynamics for Q-learning with memory one in symmetric two-player, two-action games

Published in Complexity, 2022

We develop a method based on computer algebra systems to represent the mutual pure strategy best-response dynamics of symmetric two-player, two-action repeated games played by players with a one-period memory. We apply this method to the iterated prisoner’s dilemma, stag hunt, and hawk-dove games and identify all possible equilibrium strategy pairs and the conditions for their existence. The only equilibrium strategy pair that is possible in all three games is the win-stay, lose-shift strategy. Lastly, we show that the mutual best-response dynamics are realized by a sample batch Q-learning algorithm in the infinite batch size limit.

Recommended citation: Meylahn, J. M., & Janssen, L. (2022). "Limiting dynamics for Q-learning with memory one in symmetric two-player, two-action games" Complexity . 0(0). https://doi.org/10.1155/2022/4830491

Intrinsic fluctuations of reinforcement learning promote cooperation

Published in Scientific Reports, 2023

In this work, we ask for and answer what makes classical temporal-difference reinforcement learning with 𝜖 -greedy strategies cooperative. Cooperating in social dilemma situations is vital for animals, humans, and machines. While evolutionary theory revealed a range of mechanisms promoting cooperation, the conditions under which agents learn to cooperate are contested. Here, we demonstrate which and how individual elements of the multi-agent learning setting lead to cooperation. We use the iterated Prisoner’s dilemma with one-period memory as a testbed. Each of the two learning agents learns a strategy that conditions the following action choices on both agents’ action choices of the last round. We find that next to a high caring for future rewards, a low exploration rate, and a small learning rate, it is primarily intrinsic stochastic fluctuations of the reinforcement learning process which double the final rate of cooperation to up to 80%. Thus, inherent noise is not a necessary evil of the iterative learning process. It is a critical asset for the learning of cooperation. However, we also point out the trade-off between a high likelihood of cooperative behavior and achieving this in a reasonable amount of time. Our findings are relevant for purposefully designing cooperative algorithms and regulating undesired collusive effects.

Recommended citation: Barfuss, W., & Meylahn, J. M. (2023). "Intrinsic fluctuations of reinforcement learning promote cooperation" Scientific Reports . 13(1309). https://doi.org/10.1038/s41598-023-27672-7

Reduced Plasticity in Coupling Strength in the Aging SCN Clock as Revealed by Kuramoto Modeling

Published in Journal of Biological Rhythms, 2023

The mammalian circadian clock is located in the suprachiasmatic nucleus (SCN) and consists of a network of coupled neurons, which are entrained to the environmental light-dark cycle. The phase coherence of the neurons is plastic and driven by the duration of daylight. With aging, the capacity to behaviorally adapt to seasonal changes in photoperiod reduces. The mechanisms underlying photoperiodic adaptation are largely unknown, but are important to unravel for the development of novel interventions to improve the quality of life of the elderly. We analyzed the phase coherence of single-cell PERIOD2::LUCIFERASE (PER2::LUC) expression rhythms in the SCN of young and old mice entrained to either long or short photoperiod. The phase coherence was used as input to a 2-community noisy Kuramoto model to estimate the coupling strength between and within neuronal subpopulations. The model revealed a correlation between coupling strength and photoperiod-induced changes in the phase relationship among neurons, suggesting a functional link. We found that the SCN of young mice adapts in coupling strength over a large range, with weak coupling in long photoperiod (LP) and strong coupling in short photoperiod (SP). In aged mice, we also found weak coupling in LP, but a reduced capacity to reach strong coupling in SP. The inability to respond with an increase in coupling strength suggests that manipulation of photoperiod is not a suitable strategy to enhance clock function with aging. We conclude that the inability of aged mice to reach strong coupling contributes to deficits in behavioral adaptation to seasonal changes in photoperiod.

Recommended citation: van Beurden A.W., & Meylahn J.M., & Achterhof S., & et al. (2023). "Reduced Plasticity in Coupling Strength in the Aging SCN Clock as Revealed by Kuramoto Modeling" Journal of Biological Rhythms . 38(5):461-475. https://doi.org/10.1177/07487304231175191

talks

Hierarchical synchronization with the noisy Kuramoto model

Published: May 19, 2019

A variational approach to network games

Published: May 27, 2021

Learning to collude in a pricing duopoly

Published: July 14, 2021

Can Decentralized Q-learning learn to collude?

Published: October 30, 2024

During this poster presentation, I cover two recent papers on quantifying the liklihood with which multiagent reinforcement learning algorithms can learn to collude in two simple pricing environments. The first develops the theory and applies it to two-player, two-action games and can be downloaded here. The second applies the theory to a pricing environment with three prices and can be downloaded here.

Janusz Meylahn

Sitemap

Pages

Posts

portfolio

publications

talks

teaching