引言
BYOL uses two neural networks, referred to as online and target networks, that interact and learn from each other.
Starting from an augmented view of an image, BYOL trains its online network to predict the target network’s representation of another augmented view of the same image.
相关工作
提到DeepCluster，没有负例，通过bootstrap逐渐学习。
Our approach has some similarities with Predictions of Bootstrapped Latents (PBL), a self-supervised representation learning technique for reinforcement learning (RL). The observation encoding is used as a target to train the agent’s representation, and the agent’s representation as a target to train the observation encoding. PBL does not require a second network.
然后再提RL，The idea of using a slow-moving average target network to produce stable targets for the online network was inspired by deep RL. While most RL methods use fixed target networks, BYOL uses a weighted moving average of previous networks in order to provide smoother changes in the target representation.
方法
the core motivation for BYOL: from a given representation, referred to as target, we can train a new, potentially enhanced representation, referred to as online, by predicting the target representation.
（OnlineNet先以TargetNet学习）
From there, using subsequent online networks as new target networks for further training.
（TargetNet再学习OnlineNet）
