automerge/automerge

Join the Automerge Slack community

Automerge is a library of info constructions for building collaborative functions in JavaScript.

A conventional manner to building JavaScript apps entails keeping the declare of your software program in
model objects, similar to a JSON doc. As an illustration, factor in you is probably to be setting up a role-tracking app
wherein each and every job is represented by a card. In vanilla JavaScript it’s good to possibly well write the following:

var doc = {cards: []}

// User provides a card
doc.cards.push({title: 'Reticulate splines', performed: counterfeit})

// User marks a role as performed
doc.cards[0].performed = correct

// Save the doc to disk
localStorage.setItem('MyToDoList', JSON.stringify(doc))

Automerge is feeble in a an analogous method, nonetheless the reliable distinction is that it helps computerized syncing
and merging:

• You should well presumably even personal a copy of the software program declare within the neighborhood on loads of devices (that can well presumably belong to the
  same consumer, or to different customers). Every consumer can independently update the software program declare on
  their native instrument, even while offline, and save the declare to native disk.

  (Identical to git, which enables you to edit files and commit changes offline.)

• When a community connection is available, Automerge figures out which changes favor to be synced from
  one instrument to any other, and brings them into the same declare.

  (Identical to git, which ability that you just can push your individual changes, and pull changes from other developers,
  in case you is probably to be online.)

• If the declare used to be similtaneously modified on different devices, Automerge automatically merges the
  changes collectively cleanly, so as that all people ends within the same declare, and no changes are misplaced.

  (Diversified from git: no merge conflicts to resolve!)

Components and Get Tips

• Community-agnostic. Automerge is a pure info structure library that would now not care what roughly
  community you utilize: client/server, come at some stage in-to-come at some stage in, Bluetooth, provider pigeon, whatever, the relaxation goes.
  Bindings to particular networking technologies are dealt with by separate libraries. As an illustration, look
  MPL for an implementation that uses Automerge in a
  come at some stage in-to-come at some stage in model utilizing WebRTC.
• Immutable declare. A Automerge object is an immutable snapshot of the software program declare at one
  deadline. Everytime you perform a switch, or merge in a switch that came from the community, you
  get merit a brand novel declare object reflecting that switch. This truth makes Automerge effectively matched with the
  functional reactive programming sort of Redux and
  Elm, as an illustration. Internally, Automerge is built upon Fb’s
  Immutable.js, nonetheless the Automerge API uses fashionable
  JavaScript objects (utilizing
  Object.freeze
  to conclude accidental mutation).
• Automatic merging. Automerge is a so-known as Warfare-Free Replicated Records Style
  (CRDT), which enables
  concurrent changes on different devices to be merged automatically without requiring any central
  server. It’s according to tutorial study on JSON CRDTs, nonetheless
  the small print of the algorithm in Automerge are different from the JSON CRDT paper, and we’re
  planning to submit extra detail about it in some unspecified time in the future.
• Fairly portable. We’re now now not yet making an effort to toughen outdated school platforms, nonetheless we now personal tested
  Automerge in Node.js, Chrome, Firefox, and Electron.

Setup

In the occasion you is probably to be in Node.js, it’s good to possibly well also set up Automerge via npm:

$ npm set up --save automerge

Then it’s good to possibly well also import it with require('automerge') as within the instance under.

Otherwise, clone this repository, and you then can also use the following instructions:

• npm set up — installs dependencies.
• npm test — runs the test suite in Node.
• npm bustle browsertest — runs the test suite in internet browsers.
• npm bustle webpack — creates a bundled JS file dist/automerge.js for internet browsers.
  It entails the dependencies and is determined up so as that it’s good to possibly well presumably be also load via a script label.

Instance Utilization

The next code samples give a transient overview of techniques to utilize Automerge.
For an example of a proper-life software program built upon Automerge, test out
Trellis, a venture management software program.

// That is how you load Automerge in Node. In a browser, simply at the side of the
// script label will organize the Automerge object.
const Automerge = require('automerge')

// For instance doc1 is the software program declare on instrument 1.
// Further down we will simulate a 2nd instrument.
let doc1 = Automerge.init()

// That preliminary declare is lawful an empty object: {}
// If truth be told, it's got an automatically generated _objectId property, nonetheless we will
// cross over the thing IDs from this case in characterize to perform it more uncomplicated to
// study.

// The doc1 object is immutable -- it's good to possibly well now now not switch it straight (in case you try,
// you will both get an exception or your switch will probably be silently disregarded,
// looking on your JavaScript engine). To replace it, it's top to name
// Automerge.switch() with a callback wherein it's good to possibly well also mutate the declare. You
// can also furthermore consist of a human-readable description of the switch, admire a commit
// message, which is saved within the switch history (look under).

doc1 = Automerge.switch(doc1, 'Initialize card listing', doc => {
  doc.cards = []
})

// { cards: [] }

// To replace the declare, it's good to possibly well also use the fashionable JavaScript array mutation
// techniques similar to push(). Internally, Automerge translates this mutable API
// name into an update of the immutable declare object. Reveal that we must always cross in
// doc1, and get merit an up so some distance object which we place to the same variable
// doc1. The fashioned doc object is now now not modified.

doc1 = Automerge.switch(doc1, 'Add card', doc => {
  doc.cards.push({title: 'Rewrite all the pieces in Clojure', performed: counterfeit})
})

// { cards: [ { title: 'Rewrite everything in Clojure', done: false } ] }

// Automerge also defines an insertAt() method for inserting a brand novel ingredient at a particular
// discipline in a listing. That it's good to possibly equally effectively use splice(), in case you tackle.
doc1 = Automerge.switch(doc1, 'Add any other card', doc => {
  doc.cards.insertAt(0, {title: 'Rewrite all the pieces in Haskell', performed: counterfeit})
})

// { cards:
//    [{title:'RewriteallthepiecesinHaskell'performed:counterfeit}
//      { title: 'Rewrite all the pieces in Clojure', performed: counterfeit } ] }

// Now let's simulate any other instrument, whose software program declare is doc2. We
// initialise it separately, and merge doc1 into it. After merging, doc2 has
// a copy of the total cards in doc1.

let doc2 = Automerge.init()
doc2 = Automerge.merge(doc2, doc1)

// Now perform a switch on instrument 1:
doc1 = Automerge.switch(doc1, 'Tag card as performed', doc => {
  doc.cards[0].performed = correct
})

// { cards:
//    [{title:'RewriteallthepiecesinHaskell'performed:correct}
//      { title: 'Rewrite all the pieces in Clojure', performed: counterfeit } }

// And, unbeknownst to instrument 1, also perform a switch on instrument 2:
doc2 = Automerge.switch(doc2, 'Delete card', doc => {
  delete doc.cards[1]
})

// { cards: [ { title: 'Rewrite everything in Haskell', done: false } ] }

// Now comes the moment of truth. Let's merge the changes from instrument 2 merit
// into instrument 1. You should well presumably also furthermore plot the merge the mistaken method round, and likewise you will get
// the same consequence. The merged consequence remembers that 'Rewrite all the pieces in
// Haskell' used to be build to correct, and that 'Rewrite all the pieces in Clojure' used to be
// deleted:

let finalDoc = Automerge.merge(doc1, doc2)

// { cards: [ { title: 'Rewrite everything in Haskell', done: true } ] }

// As our final trick, we are able to stare the switch history. Automerge
// automatically keeps music of each and every switch, alongside with the "commit message"
// that you just passed to change(). In the occasion you are expecting that history, it entails both
// changes you made within the neighborhood, and likewise changes that came from other devices. You
// can also furthermore look a snapshot of the software program declare at any moment in time within the
// previous. As an illustration, we are able to depend what number of cards there had been at each and every point:

Automerge.getHistory(finalDoc)
  .method(declare => [declare.switch.message, declare.snapshot.cards.length])
// [ [ 'Initialize card list', 0 ],
//   [ 'Add card', 1 ],
//   [ 'Add another card', 2 ],
//   [ 'Mark card as done', 2 ],
//   [ 'Delete card', 1 ] ]

Documentation

Automerge doc lifecycle

Automerge.init(actorId) creates a brand novel, empty Automerge doc.
You should well presumably also optionally cross in an actorId, which is a string that uniquely identifies the novel
node; in case you cross over actorId, a random UUID is generated.

In the occasion you cross in your individual actorId, it’s top to perform definite that there can never be two different processes
with the same actor ID. Even in case it’s good to possibly well also simply personal got two different processes running on the same machine, they
must personal definite actor IDs. Unless you realize what you is probably to be doing, it’s counseled that you just stick
with the default, and let actorId be auto-generated.

Automerge.save(doc) serializes the declare of Automerge doc doc to a string, which it’s good to possibly well also
write to disk. The string comprises an encoding of the total full switch history of the doc
(a chunk admire a git repository).

Automerge.load(string, actorId) unserializes an Automerge doc from a string that used to be
produced by Automerge.save(). The actorId argument is optional, and enables you to specify
a string that uniquely identifies the novel node, admire with Automerge.init(). Unless you realize
what you is probably to be doing, it’s counseled that you just cross over the actorId argument.

Manipulating and inspecting declare

Automerge.switch(doc, message, callback) ability that you just can alter an Automerge doc doc.
The doc object is now now not modified straight, because it’s miles immutable; as a change, Automerge.switch()
returns an up so some distance reproduction of the doc. The callback just generally known as with a mutable reproduction of
doc, as confirmed under. The message argument enables you to glue arbitrary additional info to the
switch, which is now now not interpreted by Automerge, nonetheless saved as phase of the switch history. The message
argument is optional; in case it’s good to possibly well presumably presumably admire to cross over it, it’s good to possibly well also simply name Automerge.switch(doc, callback).

Interior the callback it’s good to possibly well also use frequent JavaScript object manipulation operations to change the
doc:

newDoc = Automerge.switch(currentDoc, doc => {
  doc.property    = 'label'  // assigns a string label to a property
  doc['property'] = 'label'  // equal to the outdated line

  delete doc['property']     // removes a property

  doc.stringValue = 'label'  // all JSON outdated datatypes are supported
  doc.numberValue = 1
  doc.boolValue = correct
  doc.nullValue = null

  doc.nestedObject = {}      // creates a nested object
  doc.nestedObject.property = 'label'

  // it's good to possibly well also furthermore place an object that already has some properties:
  doc.otherObject = {key: 'label', amount: forty two}
})

Object properties starting with an underscore can now now not be feeble, as these are reserved by Automerge.

The cease-level Automerge doc is mostly an object (i.e. a mapping from properties to values).
You should well presumably also use arrays (lists) by assigning a JavaScript array object to a property within a doc.
Then it’s good to possibly well also use loads of the frequent
Array functions
to govern the array:

newDoc = Automerge.switch(currentDoc, doc => {
  doc.listing = []              // creates an empty listing object
  doc.listing.push(2, Three)        // push() provides choices to the cease
  doc.listing.unshift(0, 1)     // unshift() provides choices at the starting
  doc.listing[Three] = Math.PI      // overwriting listing ingredient by index
  // now doc.listing is [0, 1, 2, 3.141592653589793]

  // Looping over lists works as you'd query:
  for (let i = 0; i < doc.listing.length; i++) doc.listing[i] *= 2
  // now doc.listing is [0, 2, 4, 6.283185307179586]

  doc.listing.insertAt(1, 'hey', 'world')  // inserts choices at given index
  doc.listing.deleteAt(5)                    // deletes ingredient at given index
  // now doc.listing is [0, 'hello', 'world', 2, 4]

  doc.listing.splice(2, 2, 'automerge')      // admire JS frequent Array.splice()
  // now doc.listing is [0, 'hello', 'automerge', 4]

  doc.listing[four] = {key: 'label'}  // objects can even be nested interior lists as effectively
})

The newDoc returned by Automerge.switch() is a fashionable JavaScript object containing the total
edits you made within the callback. Any parts of the doc that you just did now not switch are carried over
unmodified. The supreme particular things about it are:

• All objects within the doc are made immutable utilizing
  Object.freeze(),
  to be definite you develop now now not by likelihood adjust them outdoors of an Automerge.switch() callback.
• Every object and every array has an _objectId property, which is feeble by Automerge to music
  which object is which.
• Objects even personal a _conflicts property, which is feeble when loads of customers perform conflicting
  changes at the same time (look under).

Text modifying toughen

Automerge.Text offers experimental toughen for collaborative textual explain material modifying.
Beneath the hood, textual explain material is represented as a listing of characters, which is edited by inserting or
deleting person characters. When in contrast to utilizing a fashionable JavaScript array,
Automerge.Text offers higher performance.

(Aspect exclaim: technically, textual explain material must unexcited be represented as a listing of
Unicode grapheme clusters.
What the patron thinks of as a « persona » can also simply in actuality be a series of loads of Unicode code aspects,
at the side of accents, diacritics, and other combining marks. A grapheme cluster is the smallest
editable unit of textual explain material: that’s, the thing that will get deleted in case you press the delete key as soon as, or the
thing that the cursor skips over in case you press the high-quality-arrow key as soon as. Emoji perform a accurate test case,
since many emoji encompass a chain of loads of Unicode code aspects — as an illustration, the
skintone modifier is a combining heed.)

You should well presumably also construct a Text object interior a switch callback.
Then it’s good to possibly well also use insertAt() and deleteAt() to insert and delete characters (same API as for
listing changes, confirmed above):

newDoc = Automerge.switch(currentDoc, doc => {
  doc.textual explain material = novel Automerge.Text()
  doc.textual explain material.insertAt(0, 'h', 'e', 'l', 'l', 'o')
  doc.textual explain material.deleteAt(0)
  doc.textual explain material.insertAt(0, 'H')
})

To stare a textual explain material object and render it, it’s good to possibly well also use the following techniques
(outdoors of a switch callback):

newDoc.textual explain material.length   // returns 5, the series of characters
newDoc.textual explain material.get(0)   // returns 'H', the 0th persona within the textual explain material
newDoc.textual explain material.be a half of('') // returns 'Good day', the concatenation of all characters
for (let char of newDoc.textual explain material) console.log(char) // iterates over all characters

Sending and receiving changes

The Automerge library itself is agnostic to the community layer — that’s, it’s good to possibly well also use whatever
communication mechanism you admire to get changes from one node to any other. There are currently
a pair of alternate choices, with extra under improvement:

• Expend Automerge.getChanges() and Automerge.applyChanges() to manually capture changes on one
  node and observe them on any other.
• Expend Automerge.Connection,
  an implementation of a protocol that syncs up two nodes by determining missing changes and
  sending them to one any other.
• Expend MPL, which runs the Automerge.Connection protocol
  over WebRTC.

The getChanges()/applyChanges() API works as follows:

// On one node
newDoc = Automerge.switch(currentDoc, doc => {
  // perform arbitrary switch to the doc
})
val changes = Automerge.getChanges(currentDoc, newDoc)
community.broadcast(JSON.stringify(changes))

// On any other node
val changes = JSON.parse(community.receive())
newDoc = Automerge.applyChanges(currentDoc, changes)

Reveal that Automerge.getChanges(oldDoc, newDoc) takes two documents as arguments: an outdated school declare
and a brand novel declare. It then returns a listing of the total changes that had been made in newDoc since
oldDoc. In the occasion you want a listing of the total changes ever made in newDoc, it’s good to possibly well also name
Automerge.getChanges(Automerge.init(), newDoc).

The counterpart, Automerge.applyChanges(oldDoc, changes) applies the listing of changes to the
given doc, and returns a brand novel doc with those changes applied. Automerge ensures that
on every occasion any two documents personal applied the same build of changes — although the changes had been
applied in a definite characterize — then those two documents are equal. That property generally known as
convergence, and it’s the essence of what Automerge is all about.

Automerge.merge(doc1, doc2) is a linked just that’s pleasant for testing. It appears to be like for any
changes that seem in doc2 nonetheless now now not in doc1, and applies them to doc1, returning an up so some distance
version of doc1. This just requires that doc1 and doc2 personal different actor IDs (that’s,
they originated from different calls to Automerge.init()). Peep the Instance Utilization piece above
for an example utilizing Automerge.merge().

Conflicting changes

Automerge enables different nodes to independently perform arbitrary changes to their respective copies
of a doc. Typically, those changes can even be blended with none disaster. As an illustration, if
customers adjust two different objects, or two different properties within the same object, then it’s miles
easy to mix those changes.

If customers similtaneously insert or delete items in a listing (or characters in a textual explain material doc), Automerge
preserves the total insertions and deletions. If two customers similtaneously insert at the same discipline,
Automerge will arbitrarily location one of the most insertions first and the opposite 2nd, while making certain
that the final characterize is an analogous on all nodes.

The supreme case Automerge can now now not tackle automatically, due to there’s no effectively-defined dedication,
is when customers similtaneously update the same property within the same object (or, equally, the same
index within the same listing). In this case, Automerge arbitrarily picks one of the most similtaneously written
values as the « winner »:

let doc1 = Automerge.switch(Automerge.init(), doc => { doc.x = 1 })
let doc2 = Automerge.switch(Automerge.init(), doc => { doc.x = 2 })
doc1 = Automerge.merge(doc1, doc2)
doc2 = Automerge.merge(doc2, doc1)
// Now, doc1 would be both {x: 1} or {x: 2} -- the possibility is random.
// Alternatively, doc2 would be the same, whichever label is chosen as winner.

Even supposing handiest one of the most similtaneously written values shows up within the thing, the opposite values are
now now not misplaced. They are merely relegated to a _conflicts object:

doc1 // {x: 2}
doc2 // {x: 2}
doc1._conflicts // {x: {'0506162a-ac6e-4567-bc16-a12618b71940': 1}}
doc2._conflicts // {x: {'0506162a-ac6e-4567-bc16-a12618b71940': 1}}

Right here, the _conflicts object comprises the property x, which goes the title of the property
on which the concurrent assignments came about. The nested key 0506162a-ac6e-4567-bc16-a12618b71940
is the actor ID that performed the project, and the associated label is the price it assigned
to the property x. You’ll have to use the tips within the _conflicts object to explain the warfare
within the patron interface.

The subsequent time you place to a conflicting property, the warfare is automatically view about to
be resolved, and the property disappears from the _conflicts object.

Analyzing doc history

An Automerge doc internally saves a entire history of the total changes that had been ever made
to it. This allows a fine feature: taking a study the doc declare at previous aspects in time, a.good ample.a.
time dawdle!

Automerge.getHistory(doc) returns a listing of all edits made to a doc. Every edit is an object
with two properties: switch is the inner representation of the switch (within the same fabricate as
Automerge.getChanges() returns), and snapshot is the declare of the doc today lawful
after that switch had been applied.

Automerge.getHistory(doc2)
// [{switch:{message:'Planxto1'}snapshot:{x:1}}
//   { switch: { message: 'Plan x to 2', ... }, snapshot: { x: 2 } } ]

Interior the switch object, the property message is determined to the free-fabricate « commit message » that
used to be passed in as 2nd argument to Automerge.switch() (if any). The remainder of the switch object
is particular to Automerge implementation details, and on the total wouldn’t favor to be interpreted.

In the occasion it’s top to search out out what in actuality modified in a particular edit, in preference to inspecting the
switch object, it’s miles extra healthy to utilize Automerge.diff(oldDoc, newDoc). This just returns a listing
of edits that had been made in doc newDoc since its prior version oldDoc. You should well presumably also cross in
snapshots returned by Automerge.getHistory() in characterize to search out out variations between historical
versions.

The tips returned by Automerge.diff() has the following fabricate:

let history = Automerge.getHistory(doc2)
Automerge.diff(history[2].snapshot, doc2) // get all changes since history[2]
// [{movement:'build'sort:'method'obj:''key:'x'label:1}
//   { movement: 'build', sort: 'method', obj: '...', key: 'x', label: 2 } ]

In the objects returned by Automerge.diff(), obj indicates the thing ID of the thing being
edited (matching its _objectId property), and sort indicates whether that object is a method,
listing, or textual explain material.

The obtainable values for movement depend upon the sort of object. For sort: 'method', the imaginable
actions are:

• movement: 'build': Then the property key is the title of the property being up so some distance. If the price
  assigned to the property is a outdated (string, amount, boolean, null), then label comprises
  that label. If the assigned label is an object (method, listing, or textual explain material), then label comprises the
  _objectId of that object, and furthermore the property hyperlink: correct is determined. Moreover, if this
  project prompted conflicts, then the conflicting values are furthermore contained in a
  conflicts property.
• movement: 'carry': Then the property key is the title of the property being eliminated.

For sort: 'listing' and sort: 'textual explain material', the imaginable actions are:

• movement: 'insert': Then the property index comprises the listing index at which a brand novel ingredient is
  being inserted, and label comprises the price inserted there. If the inserted label is an
  object, the label property comprises its _objectId, and the property hyperlink: correct is determined.
• movement: 'build': Then the property index comprises the listing index to which a brand novel label is being
  assigned, and label comprises that label. If the assigned label is an object, the label
  property comprises its _objectId, and the property hyperlink: correct is determined.
• movement: 'carry': Then the property index comprises the listing index that’s being eliminated from
  the listing.

Caveats

The venture currently has a series of limitations that it’s top to unexcited personal in tips of:

• No integrity checking: if a buggy (or malicious) instrument makes corrupted edits, it will build off
  the software program declare on other devices to be come corrupted or saunter out of sync.
• No security: there is currently no encryption, authentication, or get admission to manage.
• Little series of collaborators: Automerge is designed for small-neighborhood collaborations. Whereas there
  just isn’t any laborious restrict on the series of devices that can well update a doc, performance will degrade
  in case you saunter beyond, disclose, a hundred devices or so.
• …and extra, look the open concerns.

Meta

Copyright 2017, Ink & Swap LLC, and College of Cambridge.
Launched under the terms of the MIT license (look LICENSE).

Created by
Martin Kleppmann,
Orion Henry,
Peter van Hardenberg,
Roshan Choxi, and
Adam Wiggins.