8.1 Persistence tutorial

When writing Goblins code without considering persistence, we simply define an outer procedure (the constructor, usually ^my-constructor) and an inner procedure (the behavior, most often using an anonymous lambda) that’s called when the object receives a message. Persistence capable objects are a little different; while they still follow this general pattern, they also need to provide a way to describe themselves. Fortunately, Goblins provides the define-actor syntax which makes this easier. For more advanced cases where define-actor is unsuitable, see Persistence capable objects for the lower-level details of creating persistent objects.

Let’s start with the cell example from the Tutorial:

(define* (^cell bcom #:optional (val #f))
  (case-lambda
    (()         ; get
     val)
    ((new-val)  ; set
     (bcom (^cell bcom new-val)))))

To make ^cell persistence capable, define can be changed to define-actor:

(use-modules (goblins))

(define-actor (^cell bcom #:optional (val #f))
  (case-lambda
    (()         ; get
     val)
    ((new-val)  ; set
     (bcom (^cell bcom new-val)))))

And it’s done! Now, let’s try it out. To use persistence, there’s an additional prerequisite: a persistence environment must be defined. “What is this extra thing?”, I hear you ask.

In short, serializing an object graph requires translating each object from a Scheme procedure representing its current behavior (which cannot be serialized) to a serializable self-portrait. Goblins does this by storing each object as a mapping of labels to self-portraits. In the ^cell constructor above, define-actor has taken care of the self-portrait for us.

To restore an object graph from its serialized form, a persistence environment is required. A persistence environment is simply a mapping between labels and constructor procedures. To demonstrate, let’s define a simple persistence environment that only has our shiny new persistent ^cell in it:

(define my-env
  (make-persistence-env
   `((((my-cool-scheme-project cell) ^cell) ,^cell))))

;;    |-----------------------------------| |----|
;;                    label                  proc

Just remember that each label must be unique within the environment, or else you’ll get into quite the pickle (for more information, see Persistence environments).

The persistence tools work with both Vats and the lower-level Actormaps (since vats are built on top of actormaps). Persistent vats require a store: a place for the vat to read and write serialized object graph data. One such store is the Memory store, which is ephemeral and not actually saved to any kind of long-term storage. A persistent vat with a memory store can be spawned like this:

(use-modules (goblins)
             (goblins persistence-store memory))

(define memory-store (make-memory-store))
(define-values (my-vat my-cell)
  (spawn-persistent-vat
    my-env
    (lambda ()
      (spawn ^cell))
    memory-store))

Hold on, did we just spawn a ^cell object while spawning the vat?

Yes, we did! Notably, this is not necessary for all the objects in the graph, just the roots (the objects that spawn all the other objects). When spawn-persistent-vat is called it first checks in the store to see if there’s any saved data. If not then the thunk (the lambda with no arguments above) is called to spawn the root objects for the first time. If there is saved data then it is deserialized and the associated persistence environment is used to restore the object graph.

Since the store is empty initially, the value of my-cell is a reference to a freshly spawned cell:

> ,enter-vat my-vat
goblins[1]> ($ my-cell 'apple)

Now that the persistent vat and its roots have spawned, the object graph has automatically been saved to the store. By default, a persistent vat will continuously update the store upon changes to object state.

When debugging it can be useful to see the serialized form of an object. To do this from the REPL, use the vat-take-object-portrait metacommand:

goblins[1]> ,vat-take-object-portrait my-cell
(apple)

Now, let’s look at a more advanced example of a persistent object. We’ll first build it without persistence in mind and then see some of the problems we might encounter when making an object persistable. Below is an object representing a player ship in a space shooter video game.

(use-modules (goblins actor-lib methods))

(define (^player-ship bcom init-x init-y)
  (define x (spawn ^cell init-x))
  (define y (spawn ^cell init-y))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) ($ x))
     ((y) ($ y))
     ((fire) "pew pew")
     ((kill) (bcom dead-beh))
     ((tick) ($ x (1+ ($ x))))))
  active-beh)

This ship object contains the (x, y) coordinates for its current position. These coordinates are stored in cells so that the position can be changed over time. The ship spawns in the “active” state and responds to several methods:

• x and y Get the current position.
• fire Shoot, or rather just say “pew pew”.
• kill Transitions the object to the “dead” state.
• tick Move the forward by 1 unit on the X axis. This method could be hooked up to a game loop to animate the player.

To make the ship persist, we could try simply changing define to define-actor:

(define-actor (^player-ship bcom init-x init-y)
  (define x (spawn ^cell init-x))
  (define y (spawn ^cell init-y))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) ($ x))
     ((y) ($ y))
     ((fire) "pew pew")
     ((kill) (bcom dead-beh))
     ((tick) ($ x (1+ ($ x))))))
  active-beh)

At first it might seem like that was enough, but there are actually some big problems when we try it out:

goblins[1]> (define ship (spawn ^player-ship 0 0))
goblins[1]> ,vat-take-object-portrait ship
(0 0)

So far, so good. Now let’s try ticking the game loop to see if the X coordinate changes:

goblins[1]> ($ ship 'tick)
goblins[1]> ,vat-take-object-portrait ship
(0 0)

Hmm, that’s not right. Well, what happens if we kill the player?

goblins[1]> ($ ship 'kill)
goblins[1]> ,vat-take-object-portrait ship
(0 0)

Okay, that’s the same as before. Doesn’t seem like the object would be restored in the dead state. What’s happening?

The problem is that the ^player-ship constructor takes two arguments (the initial X and Y coordinates) but then spawns cells internally. These cells live within the closure of the object and are thus inaccessible to the persistence system (and anything else, really). We need a constructor that accepts all of the values that are needed to serialize a complete self-portrait. The new player ship definition below does exactly that:

(define-actor (^player-ship bcom x y #:key (active? #t))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) x)
     ((y) y)
     ((fire) "pew pew")
     ((kill) (bcom (^player-ship bcom x y #:active? #f)))
     ((tick) (bcom (^player-ship bcom (1+ x) y)))))
  (if active?
      active-beh
      dead-beh))

Let’s test it out:

goblins[1]> (define ship (spawn ^player-ship 0 0))
goblins[1]> ,vat-take-object-portrait ship
(0 0 #:active? #t)

Looks good so far. Now let’s try ticking it:

goblins[1]> ($ ship 'tick)
goblins[1]> ,vat-take-object-portrait ship
(1 0 #:active? #t)

Great! Finally, let’s make sure we can kill it:

goblins[1]> ($ ship 'kill)
goblins[1]> ,vat-take-object-portrait ship
(1 0 #:active? #f)

Everything works!

Instead of mutating internal cells, we simply pass updated values to the constructor using bcom and now define-actor’s built-in self-portrait contains all of the necessary data to restore the player ship later. In summary, when using define-actor we need to think of both the behavior and state of an object as being determined by the constructor rather than relying upon internal state.

There are many other ways to implement this object. Let’s look at two more and then put these different ideas together to create a much more advanced player ship.

First, we don’t have to do away with cells. We could pass cells into the constructor directly, but then we would lose some of the ergonomics from the previous version of the constructor; users would have to manually spawn these cells. Here’s a solution that continues to use cells but keeps the old interface:

(use-modules (goblins actor-lib cell))

;; IMPORTANT: note this is now ^player-ship* not ^player-ship.
(define-actor (^player-ship* bcom x y #:key (active? #t))
  (define (dead-beh)
    (error "I'm already dead!"))
  (define active-beh
    (methods
     ((x) ($ x))
     ((y) ($ y))
     ((fire) "pew pew")
     ((kill) (bcom (^player-ship bcom x y #:active? #f)))
     ((tick) ($ x (1+ ($ x))))))
  (if active?
      active-beh
      dead-beh))

(define (^player-ship bcom x y)
  (spawn ^player-ship*
         (spawn ^cell x)
         (spawn ^cell y)))

;; NOTE: Because of the feature of returning an object described
;; below, only ^player-ship* needs to be in the environment.
(define my-env
  (make-persistence-env
    `((((game) ^player-ship) ,^player-ship*))
    #:extends cell-env))

This wrapper pattern involves a public constructor with the desired interface that does some setup and then calls a private constructor. In this case, the public interface both prevents a user from spawning a dead ship and handles the “cellification” of the X and Y coordinates.

A typical constructor returns a procedure that is used as the object’s initial behavior. However, it’s also possible to return a reference to another object! Instead of creating a new object, Goblins will simply return that reference to the caller of the constructor.

Despite ^player-ship being a wrapper around ^player-ship*, both procedures are defined at the top-level. In fact, this pattern would not work if ^player-ship* was encapsulated within ^player-ship. This is because the persistence environment needs access to the private constructor for the purpose of restoration.

The new persistence environment above references only the private constructor, ^player-ship*. This is because, as explained above, the ^player-ship is not a true constructor; it merely delegates to ^player-ship*. Note also that, instead of using the version of ^cell we defined above, the example uses the one from the (goblins actor-lib cell) module. So, our persistence environments needs to extend cell-env provided by that module. Persistence environments can be composed together.

Onto the second variant. Having a sort of “state” flag passed into the constructor allows the ship to decide if the active or dead behavior should be used, but there are other ways to do this. Let’s look at a particularly interesting technique which uses the Swappable module, (goblins actor-lib swappable):

(use-modules (goblins actor-lib swappable))

;; Public constructor again, except this time spawning the swappable
;; object as well as cellifying the x and y coordinates.
(define (^player-ship bcom x y)
  (define-values (beh-proxy beh-swapper)
    ;; NOTE: Due to how swappable works, it must be spawned with an
    ;; initial reference. We'll use a placeholder one which we will
    ;; immediately switch away from.
    (swappable (spawn (lambda _ (lambda _ 'noop)))))
  ;; Swap away from the dummy placeholder above to an alive ship.
  ($ beh-swapper (spawn ^player-ship/alive
			(spawn ^cell x)
			(spawn ^cell y)
			beh-swapper))
  ;; Return the proxy.
  beh-proxy)

;; The behavior and state for the alive player ship.
(define-actor (^player-ship/alive bcom x y swapper)
  (methods
   ((tick) ($ x (1+ ($ x))))
   ((x) ($ x))
   ((y) ($ y))
   ((fire) "pew pew")
   ((kill) ($ swapper (spawn ^player-ship/dead x y swapper)))))

;; The behavior and state for the dead player ship.
(define-actor (^player-ship/dead bcom x y swapper #:optional (countdown 30))
  (methods
   ((tick)
    ;; When dead instead of moving, just wait until the countdown
    ;; has reached zero and then swap to alive.
    (if (zero? countdown)
        ($ swapper (spawn ^player-ship/alive x y swapper))
        (bcom (^player-ship/dead bcom x y swapper (1- countdown)))))
   ((x) ($ x))
   ((y) ($ y))
   ((fire) 'noop)   ; cannot fire when dead
   ((kill) 'noop))) ; cannot be killed when dead

;; Note because the swappable and cell objects are being passed in and
;; thus need to be persisted in the graph, we *MUST* include those in
;; persistence environment.
(define game-env
  (make-persistence-env
   `((((game) ^player-ship/alive) ,^player-ship/alive)
     (((game) ^player-ship/dead) ,^player-ship/dead))
   #:extends (list swappable-env cell-env)))

In the above example, the ^player-ship constructor is returning the proxy from swappable as the reference for the player ship. The proxy will forward messages to the object representing the ship’s current state. The capability to swap the state of the ship is passed to the ^player-ship/alive and ^player-ship/dead constructors. While this is probably overkill for our player ship, this example illustrates that swappable can be a powerful way to implement an object with many different possible behaviors.

It may be tempting to simplify this by removing the swapper and using bcom like (bcom (spawn ^player-ship/dead ...)), but this does not work. bcom accepts a procedure representing the new behavior, not an object reference. In other words, Goblins does not allow an object to become something that it’s not (that would be a security issue).

And that’s it for the tutorial! Now, go out and save the world!