Problem solving using rewarded STDP (aka Spirit)

Spirit v3 (2022)

Paper Sleep prevents catastrophic forgetting in spiking neural networks by forming joint synaptic weight representations, PLoS Comput Biol 18(11): e1010628, 2022. (Preprint at bioRxiv).

Story: Using sleep to integrate new memory for already trained network.
Paper itself (pdf).
Features: Multiple neural layers, pattern recognition via STDP (layer 0->1), decision making via rewarded STDP (layer 1->2)
Protocol to overcome catastrophic forgetting:
    1. Train "old" memory.
    2. Train "new" memory while letting the network to "sleep" from time to time (=allowing replay events during spontaneous activity).
Press cover:
- Jeremy Hsu: AI uses artificial sleep to learn new task without forgetting the last. NewScientist, 10 Nov 2022.
- Tatyana Woodall: Scientists Taught an AI to 'Sleep' So That It Doesn't Forget What It Learned, Like a Person. VICE Motherboard, 11 Nov 2022.
- Scott LaFee: Artificial Neural Networks Learn Better When They Spend Time Not Learning at All. UC San Diego Today, 18 Nov 2022.
  - Sourced to: Technology Networks, Science Daily, Minmin Tv Cp, US Today News, Mirage News, ECHEMI.
- Charles Q. Choi: Sleep Can Keep AI From Catastrophic Forgetting: New data-replay strategy prevents AI amnesia. IEEE Spectrum, 23 Nov 2022.
- Bob Yirka: Mimicking human sleep as a way to prevent catastrophic forgetting in AI systems. Tech Xplore, 24 Nov 2022.
- Chelsea Blankenchip: Study Finds that Sleep Prevents Catastrophic Forgetting in Spiking Neural Networks. The UCSD Guardian, 28 Nov 2022.
- Kathryn Hulick: Sleep helps AI models learn new things without forgetting old ones. ScienceNewsExplore, 22 Feb 2023.
Other languages:
- FR: Imiter le sommeil humain comme moyen de prévenir l’oubli catastrophique dans les systèmes d’IA. SIMSEO, 25 Nov 2022.
- PL: Czy AI potrzebuje snu? Tak, bo dzięki niemu uczy się lepiej. Antyweb, 14 Nov 2022.
- JP: 人工ニューラルネットワークは、全く学習しない時間を過ごすことでよりよく学習するようになる . Tech-I, 19 Nov 2022.
- CN: 如何让AI具有通用能力？新研究：让它睡觉 . Leading Research, 25 Nov 2022.
- ES: Investigadores se inspiran en el sueño humano para entrenar una IA capaz de recordar procesos. Notimérica / Europa Press, 14 Nov 2022.
- ES: La inteligencia artificial que necesita dormir y descansar. Andina Link, 17 Nov 2022.
- TR: Bilim insanları yapay zekaya "uyumayı" öğretti. Independent Türkçe, 14 Nov 2022.
- IT: Il sonno fa bene anche alle intelligenze artificiali. Focus, 14 Nov 2022.

Spirit v2 (2017)

Supplementary material accompanying paper "Multi-layer Network Utilizing Rewarded Spike Time Dependent Plasticity to Learn a Foraging Task", PLoS Comput Biol 13(9): e1005705, 2017.

Story: Using biologically plausible principles on a cellular level to solve foraging task in the maze (= pattern recognition of different shapes and proper behavioral response).
Paper itself (pdf), supplementary figures, patent.
Features: Multiple neural layers, pattern recognition via STDP (layer 0->1), decision making via rewarded STDP (layer 1->2)
Animations (or go here for raw video formats):

Untrained network
Trained network

Untrained network shows random
behaviour. Trained network shows approaching
behaviour towards horizontal bars
and aversive to vertical bars.

Read array is environment for creature movement, green are food particles (horizontal tasty, vertical toxic). Creature is blue.
Blue arrays show subsequent neuronal layers in creature's brain, (input, middle and output layer), spiking of particular neuron is displayed by the red flash.

Untrained network		Trained network
Untrained network shows random behaviour.		Trained network shows approaching behaviour towards horizontal bars and aversive to vertical bars.

Spirit v1 (2014)

Story: Solving simple foraging task via rSTDP.
Paper itself.
Features: Single neural layer, decision making via rewarded STDP (1->2)
Animation:

Back to preprints.