Let us keep things simple: make a few random training data memories for our feedforward neural network modern hopfield implementation, make some random input context data, converge the input context data to the training data memories as away of carrying our real topic this article: using ansi common lisp's sharpsign equals and sharpsign sharpsign read macros to have memories (training data) that exist with reference to the current or previous input context, which is an incredibly important insight and contribution from the ai language's ansi standard.
We have this many loosely guaranteed memories (number of training data), if the data is `3x3=9` cells and we choose `8` as the polynomial degree.
```
((lambda
(total n)
(/ (EXPT TOTAL (1- N))
(* 2 (LOG TOTAL) (! (! (- (* 2 N) 3))))))
(* 3 3) 8)
```
which comes out at `45`. `9` to the power of `8` seems like a pretty big number so we better look into this. For reference,
```
(/ most-positive-fixnum 10e17)
```
is `4.6`. So polynomials ranging one order of magnitue (one zero) higher than `10` to the power of `17` run into lisp bignum territory. Lisp's bignums compute slower than its fixnums, which could be a performance bottleneck. We seem to be well below this limit, and it gives us `45` reliable `3x3` memories. Each memory is itself an arbitrarily big list of symbols; this is not part of the feedforward neural network (hopfield implementation) training data constraint. As a quick addendum, I guess a `9x9` matrix with polynomial `5` has a reliable training data memory capacity of `2844` different `9x9` matrixes (and `6`, `77606`).
## Some random training data memories
Remember in lisp to get a *random* `random-state` you use
```
(setq *random-state* (make-random-state t))
```
you might put this in your `.eclrc` and `.sbclrc`.
For reproducibility sake here let us
```
(setq *random-state* (make-random-state 1))
(defun random-axb (rows cols success failure)
(loop
:repeat rows
:collecting
(loop :repeat cols
:for choice := (random 2)
:unless (zerop choice) :collect
success
:else :collect failure)))
```
Ah, random numbers.
```
CL-USER> (random-axb 3 2 '(yes) '(no))
(((YES) (YES)) ((YES) (NO)) ((NO) (NO)))
```
...
```
CL-USER> (random-axb 3 2 '(yes) '(no))
(((YES) (YES)) ((YES) (NO)) ((YES) (YES)))
CL-USER> (random-axb 3 2 '(yes) '(no))
(((YES) (NO)) ((NO) (YES)) ((YES) (YES)))
```
Okay, let us make two 3x3 memories.
```
CL-USER> (loop :repeat 2 :collecting (random-axb 3 3 '(foo) '()))
(((NIL NIL NIL)
((FOO) NIL (FOO))
((FOO) NIL NIL))
(((FOO) (FOO) NIL)
((FOO) (FOO) NIL)
(NIL NIL (FOO))))
CL-USER> (defparameter *memories* *)
*MEMORIES*
```
t b q h I rolled this again to get more satisfying looking memories.
## Random input context data
```
CL-USER> (random-axb 3 5 '(foo) '())
(((FOO) (FOO) (FOO) (FOO) NIL)
(NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
CL-USER> (defparameter *context* *)
*CONTEXT*
```
## Writing and reading these matrices to files for lisp sharpsign sharpsign and sharpsign equals read macros
I guess in general we file data as files, which are basically text we parse. So I am going to write a way of formatting a matrix to a file with lisp's sharpsign equals reader dispatch macro which will let us have memories that reference the current input context, each other and themselves, both statically and dynamically.
```
(defun reference-mat
(stream matrix colonp atp &optional (start 0))
(declare (ignore colonp atp))
(format stream
"(~{(~{~{#~d=~s~}~^ ~})~^~%~})"
(mapcar (lambda (y)
(mapcar
(lambda (x) (list (incf start) x))
y))
matrix)))
```
So our input context is
```
CL-USER> (format t "~v/reference-mat/" 0
*context*)
((#1=(FOO) #2=(FOO) #3=(FOO) #4=(FOO) #5=NIL)
(#6=NIL #7=(FOO) #8=(FOO) #9=NIL #10=(FOO))
(#11=NIL #12=NIL #13=NIL #14=NIL #15=(FOO)))
```
Admittedly I fixed its columns with emacs and not common lisp pretty printing. Note `v` in common lisp's `format` is "pull arguement in", mnemonically like `~^` is "escape upwards".
## Training data memory text to read referencing the input data cons pointer or values
The other half of the sharpsign equals dispatch macro is the sharpsign sharpsign dispatch macro. Consider that we are probably reading our memories out of files when we use them, which is the same in terms of lisp streams as reading from strings:
```
CL-USER> (format nil "(
~{(~{(~{~s~^ ~})~^~%~})~^
~}
)" *memories*)
"(
((NIL NIL NIL)
((FOO) NIL (FOO))
((FOO) NIL NIL))
(((FOO) (FOO) NIL)
((FOO) (FOO) NIL)
(NIL NIL (FOO)))
)"
```
You will remember these memories from earlier. But let us write (as text for the default ansi common lisp reader to read in) two more memories:
For reference again the current context for reference again
```
CL-USER> (format nil "~v/reference-mat/" 0
*context*)
"
((#1=(FOO) #2=(FOO) #3=(FOO) #4=(FOO) #5=NIL)
(#6=NIL #7=(FOO) #8=(FOO) #9=NIL #10=(FOO))
(#11=NIL #12=NIL #13=NIL #14=NIL #15=(FOO)))
"
CL-USER> (defparameter *context-string* *)
*CONTEXT-STRING*
```
```
(defparameter
*referential-memories-string*
"(
((#3# #4# #5#)
(#6# #7# #8#)
(#9# #10# #11#))
(( #.`(,@'#3#) #.`(,@'#4#) #.`(,@'#5#))
( #.`(,@'#6#) #.`(e ,@'#7# g) #.`(,@'#8#))
( #.`(,@'#9#) #.`(,@'#10#) #.`(,@'#11#)))
)")
```
The first additional memory's `#3#` means *literally the same cons as* the `#3=(FOO)` when read together the input `*context*`.
Next, and sorry it turned out so messy - I think this is my first time drawing a snail in the wild, so I can cross that off my lisp user bingo card - the memory's cells are instead new lists, different conses, that have the values at read-time of `#3=(FOO)` etc spliced into them. So we can see `#.\`(e ,@'#7# g)` will turn out to be `(E FOO G)` because `#7=(FOO)`. However altering "#7" in `*context*` will not alter the spliced values unilke the memory that just has `#7#`.
The annoying extra read macros are because I wanted to use splice, but I was not inside a backquote. So I had to use `#.` *read-eval*, `\`` *backquote*, `,@` *splice*, `'` *quote* (do not eval). Hence the mess. Splice quote is called a snail.
We wrote that as a string but we need to read it together with the input `*context*` data.
## Reading ffnn sharpsign equals input with sharpsign sharpsign training data
```
(defun read-context-with-memories
(context-stream memories-stream)
(with-input-from-string
(begin "(")
(with-input-from-string
(end ")")
(apply 'values
(read (make-concatenated-stream
begin
context-stream memories-stream
end))))))
```
## Read the training data refering into the current dl context
```
CL-USER> (with-input-from-string
(context-stream *context-string*)
(with-input-from-string
(memories-stream *referential-memories-string*)
(read-context-with-memories context-stream memories-stream)))
(((FOO) (FOO) (FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
((((FOO) (FOO) NIL) (NIL (FOO) (FOO)) (NIL (FOO) NIL))
(((FOO) (FOO) NIL) (NIL (E FOO G) (FOO)) (NIL (FOO) NIL)))
```
Put the multiple values into `*memories*` and `*context*` again.
```
CL-USER> (multiple-value-setq
(*context* *memories*)
(apply 'values /))
(((FOO) (FOO) (FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
```
# Perform an inference that changes the referential memory
## Look at current context and referential memories
```
CL-USER> (dolist (m *memories*) (mapc 'print m) (terpri))
((FOO) (FOO) NIL)
(NIL (FOO) (FOO))
(NIL (FOO) NIL)
((FOO) (FOO) NIL)
(NIL (E FOO G) (FOO))
(NIL (FOO) NIL)
NIL
CL-USER> (mapc 'print *context*)
((FOO) (FOO) (FOO) (FOO) NIL)
(NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO))
(((FOO) (FOO) (FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
```
## Inference
```
```
## Changed-context
```
```
# revised.
```
(setf *context* '(((FOO) (FOO) (bar foo) (FOO) NIL)
(NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO))))
CL-USER> (format nil "~v/reference-mat/" 0
*context*)
"((#1=(FOO) #2=(FOO) #3=(BAR FOO) #4=(FOO) #5=NIL)
(#6=NIL #7=(FOO) #8=(FOO) #9=NIL #10=(FOO))
(#11=NIL #12=NIL #13=NIL #14=NIL #15=(FOO)))"
CL-USER> (defparameter *context-string* *)
*CONTEXT-STRING*
CL-USER> (defparameter
*referential-memories-string*
"(
((#3# #4# #5#)
(#6# #7# #8#)
(#9# #10# #11#))
(( #.`(,@'#3#) #.`(,@'#4#) #.`(,@'#5#))
( #.`(,@'#6#) #.`(e ,@'#7# g) #.`(,@'#8#))
( #.`(,@'#9#) #.`(,@'#10#) #.`(,@'#11#)))
)")
*REFERENTIAL-MEMORIES-STRING*
CL-USER> (with-input-from-string
(context-stream *context-string*)
(with-input-from-string
(memories-stream *referential-memories-string*)
(read-context-with-memories context-stream memories-stream)))
(((FOO) (FOO) (BAR FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
((((BAR FOO) (FOO) NIL) (NIL (FOO) (FOO)) (NIL (FOO) NIL))
(((BAR FOO) (FOO) NIL) (NIL (E FOO G) (FOO)) (NIL (FOO) NIL)))
CL-USER> (multiple-value-setq
(*context* *memories*)
(apply 'values /))
(((FOO) (FOO) (BAR FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
```
# rerevised.
```
CL-USER> (setf *context* '(((FOO) (FOO) (bar foo) (FOO) NIL)
(NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO))))
(((FOO) (FOO) (BAR FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
CL-USER> (format nil "~v/reference-mat/" 0
*context*)
"((#1=(FOO) #2=(FOO) #3=(BAR FOO) #4=(FOO) #5=NIL)
(#6=NIL #7=(FOO) #8=(FOO) #9=NIL #10=(FOO))
(#11=NIL #12=NIL #13=NIL #14=NIL #15=(FOO)))"
CL-USER> (defparameter *context-string* *)
*CONTEXT-STRING*
CL-USER> (defparameter
*referential-memories-string*
"(
((#3# #4# #5#)
(#6# #7# #8#)
(#9# #10# #11#))
(( #.`(,@'#3#) #.`(,@'#4#) #.`(,@'#5#))
( #.`(,@'#6#) #.`(e ,@'#7# g) #.`(,@'#8#))
( #.`(,@'#9#) #.`(,@'#10#) #.`(,@'#11#)))
)")
*REFERENTIAL-MEMORIES-STRING*
CL-USER> (with-input-from-string
(context-stream *context-string*)
(with-input-from-string
(memories-stream *referential-memories-string*)
(read-context-with-memories context-stream memories-stream)))
(((FOO) (FOO) (BAR FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
((((BAR FOO) (FOO) NIL) (NIL (FOO) (FOO)) (NIL (FOO) NIL))
(((BAR FOO) (FOO) NIL) (NIL (E FOO G) (FOO)) (NIL (FOO) NIL)))
CL-USER> (multiple-value-setq
(*context* *memories*)
(apply 'values /))
(((FOO) (FOO) (BAR FOO) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
CL-USER> (test-infer 0 0 0 2 '(foo) *memories* *context*)
NEW-VALUE!
(BAR)
(BAR)
CL-USER> (nintersection (elt (elt *context* 0) 2) *)
(BAR)
CL-USER> (dolist (m *memories*) (mapc 'print m) (terpri))
((BAR) (FOO) NIL)
(NIL (FOO) (FOO))
(NIL (FOO) NIL)
((BAR) (FOO) NIL)
(NIL (E FOO G) (FOO))
(NIL (FOO) NIL)
NIL
CL-USER> (mapc 'print *context*)
((FOO) (FOO) (BAR) (FOO) NIL)
(NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO))
(((FOO) (FOO) (BAR) (FOO) NIL) (NIL (FOO) (FOO) NIL (FOO))
(NIL NIL NIL NIL (FOO)))
CL-USER>
```