Open
Conversation
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 5, 2026
See DR [1], realtime implementation [2] and spec [3] Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 5, 2026
See DR [1], realtime implementation [2] and spec [3] Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 5, 2026
See DR [1], realtime implementation [2] and spec [3] Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 5, 2026
See DR [1], realtime implementation [2] and spec [3] Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
force-pushed
the
AIT-466/map-clear
branch
from
21a5806 to
f8f4d34
Compare
RTLO4e3, RTLM6c1 and RTLM8f all defined the same logic for calculating tombstonedAt from serialTimestamp with a local clock fallback. Consolidate into a single RTLO6 spec point and replace the duplicates with references to it.
VeskeR
force-pushed
the
AIT-466/map-clear
branch
from
f8f4d34 to
001018f
Compare
Resolves AIT-466
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 6, 2026
See DR [1], realtime implementation [2] and spec [3]. The DR specifies that MAP_CLEAR is currently only emitted by the server for the root object. However, the client implementation is intentionally future-proofed to support MAP_CLEAR on any map object ID. Semantics of MAP_CLEAR support: 1. OBJECT_SYNC: The clearTimeserial from the sync state is stored on the LiveMap for use by future operations. Materialised entries from the `ObjectMessage.object.map` arrive pre-tombstoned by the server for entries that predated the clear. Initial entries from the `ObjectMessage.object.createOp` are merged via the existing MAP_SET/MAP_REMOVE semantics, which check against clearTimeserial. 2. MAP_SET: After the usual siteTimeserials check, the operation is additionally discarded if clearTimeserial is set and is lexicographically greater than or equal to the operation's serial, since the set predates (or is concurrent with) the clear. 3. MAP_REMOVE: No changes needed - a remove after a clear is valid regardless of clearTimeserial. 4. MAP_CLEAR: The operation is discarded if the current clearTimeserial is already greater than or equal to the incoming serial (stale clear). Otherwise, clearTimeserial is updated to the operation's serial, and all existing entries whose timeserial is null or less than or equal to the new clearTimeserial are tombstoned. Entries with a strictly later timeserial are preserved. Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
force-pushed
the
AIT-466/map-clear
branch
from
001018f to
de65c0f
Compare
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 6, 2026
See DR [1], realtime implementation [2] and spec [3]. The DR specifies that MAP_CLEAR is currently only emitted by the server for the root object. However, the client implementation is intentionally future-proofed to support MAP_CLEAR on any map object ID. Semantics of MAP_CLEAR support: 1. OBJECT_SYNC: The clearTimeserial from the sync state is stored on the LiveMap for use by future operations. Materialised entries from the `ObjectMessage.object.map` arrive pre-tombstoned by the server for entries that predated the clear. Initial entries from the `ObjectMessage.object.createOp` are merged via the existing MAP_SET/MAP_REMOVE semantics, which check against clearTimeserial. 2. MAP_SET: After the usual siteTimeserials check, the operation is additionally discarded if clearTimeserial is set and is lexicographically greater than or equal to the operation's serial, since the set predates (or is concurrent with) the clear. 3. MAP_REMOVE: No changes needed - a remove after a clear is valid regardless of clearTimeserial. 4. MAP_CLEAR: The operation is discarded if the current clearTimeserial is already greater than or equal to the incoming serial (stale clear). Otherwise, clearTimeserial is updated to the operation's serial, and all existing entries whose timeserial is null or less than or equal to the new clearTimeserial are tombstoned. Entries with a strictly later timeserial are preserved. Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 6, 2026
See DR [1], realtime implementation [2] and spec [3]. The DR specifies that MAP_CLEAR is currently only emitted by the server for the root object. An additional decision was made that the client should be future-proofed to support MAP_CLEAR on any map object ID, not just root. This implementation follows that decision. Semantics of MAP_CLEAR support: 1. OBJECT_SYNC: The clearTimeserial from the sync state is stored on the LiveMap for use by future operations. Materialised entries from the `ObjectMessage.object.map` arrive pre-tombstoned by the server for entries that predated the clear. Initial entries from the `ObjectMessage.object.createOp` are merged via the existing MAP_SET/MAP_REMOVE semantics, which check against clearTimeserial. 2. MAP_SET: After the usual siteTimeserials check, the operation is additionally discarded if clearTimeserial is set and is lexicographically greater than or equal to the operation's serial, since the set predates (or is concurrent with) the clear. 3. MAP_REMOVE: No changes needed - a remove after a clear is valid regardless of clearTimeserial. 4. MAP_CLEAR: The operation is discarded if the current clearTimeserial is already greater than or equal to the incoming serial (stale clear). Otherwise, clearTimeserial is updated to the operation's serial, and all existing entries whose timeserial is null or less than or equal to the new clearTimeserial are tombstoned. Entries with a strictly later timeserial are preserved. Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 6, 2026
See DR [1], realtime implementation [2] and spec [3]. The DR specifies that MAP_CLEAR is currently only emitted by the server for the root object. An additional decision was made that the client should be future-proofed to support MAP_CLEAR on any map object ID, not just root. This implementation follows that decision. Semantics of MAP_CLEAR support: 1. OBJECT_SYNC: The clearTimeserial from the sync state is stored on the LiveMap for use by future operations. Materialised entries from the `ObjectMessage.object.map` arrive pre-tombstoned by the server for entries that predated the clear. Initial entries from the `ObjectMessage.object.createOp` are merged via the existing MAP_SET/MAP_REMOVE semantics, which check against clearTimeserial. 2. MAP_SET: After the usual siteTimeserials check, the operation is additionally discarded if clearTimeserial is set and is lexicographically greater than or equal to the operation's serial, since the set predates (or is concurrent with) the clear. 3. MAP_REMOVE: No changes needed - a remove after a clear is valid regardless of clearTimeserial. 4. MAP_CLEAR: The operation is discarded if the current clearTimeserial is already greater than or equal to the incoming serial (stale clear). Otherwise, clearTimeserial is updated to the operation's serial, and all existing entries whose timeserial is null or less than or equal to the new clearTimeserial are tombstoned. Entries with a strictly later timeserial are preserved. Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 6, 2026
See DR [1], realtime implementation [2] and spec [3]. The DR specifies that MAP_CLEAR is currently only emitted by the server for the root object. An additional decision was made that the client should be future-proofed to support MAP_CLEAR on any map object ID, not just root. This implementation follows that decision. Semantics of MAP_CLEAR support: 1. OBJECT_SYNC: The clearTimeserial from the sync state is stored on the LiveMap for use by future operations. Materialised entries from the `ObjectMessage.object.map` arrive pre-tombstoned by the server for entries that predated the clear. Initial entries from the `ObjectMessage.object.createOp` are merged via the existing MAP_SET/MAP_REMOVE semantics, which check against clearTimeserial. 2. MAP_SET: After the usual siteTimeserials check, the operation is additionally discarded if clearTimeserial is set and is lexicographically greater than or equal to the operation's serial, since the set predates (or is concurrent with) the clear. 3. MAP_REMOVE: No changes needed - a remove after a clear is valid regardless of clearTimeserial. 4. MAP_CLEAR: The operation is discarded if the current clearTimeserial is already greater than or equal to the incoming serial (stale clear). Otherwise, clearTimeserial is updated to the operation's serial, and all existing entries whose timeserial is null or less than or equal to the new clearTimeserial are tombstoned. Entries with a strictly later timeserial are preserved. Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 6, 2026
See DR [1], realtime implementation [2] and spec [3]. The DR specifies that MAP_CLEAR is currently only emitted by the server for the root object. An additional decision was made that the client should be future-proofed to support MAP_CLEAR on any map object ID, not just root. This implementation follows that decision. Semantics of MAP_CLEAR support: 1. OBJECT_SYNC: The clearTimeserial from the sync state is stored on the LiveMap for use by future operations. Materialised entries from the `ObjectMessage.object.map` arrive pre-tombstoned by the server for entries that predated the clear. Initial entries from the `ObjectMessage.object.createOp` are merged via the existing MAP_SET/MAP_REMOVE semantics, which check against clearTimeserial. 2. MAP_SET: After the usual siteTimeserials check, the operation is additionally discarded if clearTimeserial is set and is lexicographically greater than or equal to the operation's serial, since the set predates (or is concurrent with) the clear. 3. MAP_REMOVE: No changes needed - a remove after a clear is valid regardless of clearTimeserial. 4. MAP_CLEAR: The operation is discarded if the current clearTimeserial is already greater than or equal to the incoming serial (stale clear). Otherwise, clearTimeserial is updated to the operation's serial, and all existing entries whose timeserial is null or less than or equal to the new clearTimeserial are tombstoned. Entries with a strictly later timeserial are preserved. Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
VeskeR
added a commit
to ably/ably-js
that referenced
this pull request
Mar 6, 2026
See DR [1], realtime implementation [2] and spec [3]. The DR specifies that MAP_CLEAR is currently only emitted by the server for the root object. An additional decision was made that the client should be future-proofed to support MAP_CLEAR on any map object ID, not just root. This implementation follows that decision. Semantics of MAP_CLEAR support: 1. OBJECT_SYNC: The clearTimeserial from the sync state is stored on the LiveMap for use by future operations. Materialised entries from the `ObjectMessage.object.map` arrive pre-tombstoned by the server for entries that predated the clear. Initial entries from the `ObjectMessage.object.createOp` are merged via the existing MAP_SET/MAP_REMOVE semantics, which check against clearTimeserial. 2. MAP_SET: After the usual siteTimeserials check, the operation is additionally discarded if clearTimeserial is set and is lexicographically greater than or equal to the operation's serial, since the set predates (or is concurrent with) the clear. 3. MAP_REMOVE: No changes needed - a remove after a clear is valid regardless of clearTimeserial. 4. MAP_CLEAR: The operation is discarded if the current clearTimeserial is already greater than or equal to the incoming serial (stale clear). Otherwise, clearTimeserial is updated to the operation's serial, and all existing entries whose timeserial is null or less than or equal to the new clearTimeserial are tombstoned. Entries with a strictly later timeserial are preserved. Resolves AIT-458 [1] https://ably.atlassian.net/wiki/x/DABECAE [2] ably/realtime#8074 [3] ably/specification#432
| ** @(RTLM24d)@ Set the private @clearTimeserial@ to the provided @serial@ | ||
| ** @(RTLM24e)@ For each @ObjectsMapEntry@ in the internal @data@: | ||
| *** @(RTLM24e1)@ If @ObjectsMapEntry.timeserial@ is null or empty, or the @serial@ is lexicographically greater than or equal to @ObjectsMapEntry.timeserial@: | ||
| **** @(RTLM24e1a)@ Set @ObjectsMapEntry.data@ to undefined/null |
Collaborator
There was a problem hiding this comment.
I'm not sure I follow why we need to tombstone these entries, instead of just removing them from the data? The RTLM7h check prevents a MAP_SET from resurrecting them.
Collaborator
There was a problem hiding this comment.
(A MAP_REMOVE for a key that didn't exist at the time of applying the MAP_CLEAR would still create an unnecessary tombstone, but that'd be addressed if we implemented https://ably-real-time.slack.com/archives/C09SY1AQGK0/p1772807443777899?thread_ts=1772801262.669639&cid=C09SY1AQGK0)
Collaborator
There was a problem hiding this comment.
Actually I think this is similar to Paddy's question here: https://ably-real-time.slack.com/archives/C09SY1AQGK0/p1772807518911119?thread_ts=1772801262.669639&cid=C09SY1AQGK0
Why aren't all entries that predated the clear just removed? I don't see why we would keep tombstones for them
| *** @(RTLM7d3)@ @MapSet@ | ||
| *** @(RTLM7d2)@ @serial@ string - operation's serial value | ||
| ** @(RTLM7e)@ The return type is a @LiveMapUpdate@ object, which indicates the data update for this @LiveMap@ | ||
| ** @(RTLM7h)@ If the private @clearTimeserial@ is non-null, and the provided @serial@ is null or the @clearTimeserial@ is lexicographically greater than or equal to @serial@, discard the operation without taking any action. Return a @LiveMapUpdate@ object with @LiveMapUpdate.noop@ set to @true@, indicating that no update was made to the object |
Collaborator
There was a problem hiding this comment.
As discussed in https://ably-real-time.slack.com/archives/C09SY1AQGK0/p1772801262669639 I think we want a similar check for MAP_REMOVE.
| * @(OME2)@ The attributes available in an @ObjectsMapEntry@ are: | ||
| ** @(OME2a)@ @tombstone@ boolean - indicates whether the map entry has been removed | ||
| ** @(OME2b)@ @timeserial@ string - the @serial@#OM2h value of the last operation that was applied to the map entry | ||
| ** @(OME2b)@ @timeserial@ string - the "serial"#OM2h value of the last operation that was applied to the map entry |
Collaborator
There was a problem hiding this comment.
Needs a colon to be a link
| * @(RTLM3)@ Holds a @Dict<String, ObjectsMapEntry>@ as a private @data@ map | ||
| ** @(RTLM3a)@ @ObjectsMapEntry@ entries in a @LiveMap@ have the following attributes in addition to those defined in "OME2":../features#OME2: | ||
| *** @(RTLM3a1)@ @tombstonedAt@ (optional) Time - a timestamp indicating when this map entry was tombstoned. This property is nullable, and specification points that manipulate this value maintain the invariant that it is non-null if and only if the corresponding @ObjectsMapEntry.tombstone@ is @true@ | ||
| * @(RTLM25)@ Holds a nullable private @clearTimeserial@ string, initially @null@. Set to the "serial":../features#OM2h of the last @MAP_CLEAR@ operation applied to this @LiveMap@ |
Collaborator
There was a problem hiding this comment.
Reads like an instruction but actually this is an invariant maintained by other specification points
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec for a MAP_CLEAR operation described in DR https://ably.atlassian.net/wiki/x/DABECAE. Realtime implementation in http://github.com/ably/realtime/pull/8074.
ably-js implementation ably/ably-js#2176
Resolves AIT-466