The University of Arizona

The need for long term memory

The need for long term memory

Series: Tripods
Location: Math 514
Presenter: Marek Rychlik, University of Arizona Mathematics

I will discuss an example of a problem motivated by optical character recognition. In this problem we study decoding messages written with characters 'X' and 'O', but we can only see the upper half of the characters. For instance, the message 'XOOXXOO' may look like 'VMWM'. This simplifying writing system has the property that it cannot be decoded by looking at a fixed window of characters, but requires remembering something from a distant past. I will also show a hand-crafted neural network which solves the decoding problem for these messages, which is a Recurrent Neural Network (RNN) with long and short term memory, and is closely related to LSTM (Long-Short Term Memory) RNNs known from literature. This example appears to be the only known example where the need for long memory is demonstrated, and yet the neural network solving the problem can be found by hand.

Department of Mathematics, The University of Arizona 617 N. Santa Rita Ave. P.O. Box 210089 Tucson, AZ 85721-0089 USA Voice: (520) 621-6892 Fax: (520) 621-8322 Contact Us © Copyright 2018 Arizona Board of Regents All rights reserved