Domino spaces

Domino Spaces

by

Arjuna C. Wijeyekoon

Submitted to the Department of Electrical Engineering and Computer

Science

in partial fulfillment of the requirements for the degree of

Master of Engineering in Computer Science and Engineering

at the

MASSACHUSETTS INSTITUTE OF TECHNOLOGY

@ Arjuna C. Wijeyekoon, MM. All rights reserved.

The author hereby grants to MIT permission to reproduce and

distribute publicly paper and electronic copies of this thesis document ENG

in whole or in part.

Author ...

JUL 2 7 2000

Department of EledricatEngineering and Computer Science

A, May 18, 2000

Certified by ...

A)

John F. Patterson

Doctor of Philosophy

Thesis Supervisor

C ertified by ...

Barbara Liskov

Professor of

Engineering

hss

Suervisor

Accepted by

Arthur C. Smith

Chairman, Department Committee on Graduate Students

Domino Spaces

by

Arjuna C. Wijeyekoon

Submitted to the Department of Electrical Engineering and Computer Science on May 18, 2000, in partial fulfillment of the

requirements for the degree of

Master of Engineering in Computer Science and Engineering

Abstract

This thesis describes a system that uses Lotus Domino and Lotus Sametime to provide synchronous collaborative access to Domino. The system is modeled after JavaSpaces. Clients may connect to a Domino Spaces server on the web and access shared per-sistent data stored in Domino databases. Clients begin working synchronously by indicating to the Domino Spaces server which data is shared and is, therefore, sen-sitive. The server will then send notifications when that data is modified, enabling the clients to keep their state consistent. The server also sends notifications about other clients participating in a collaboration. This permits clients to be aware of each other. The Domino Spaces architecture supports four types of commands - Read and Write which are used to access data, and Notify and Ignore which are used to re-quest/cancel notifications, respectively. The typing system used was general enough to allow reference to both individual objects and also to groups of objects. This allows notifications to be generated for changes to groups of data. Three applications were developed to test this implementation of Domino Spaces.

Thesis Supervisor: John F. Patterson Title: Doctor of Philosophy

Thesis Supervisor: Barbara Liskov Title: Professor of Engineering

Acknowledgments

I would like to thank Dr. John Patterson (Lotus Research) and Prof. Barbara

Liskov (MIT-LCS) for all the help, advice, and encouragement I received from them.

I am also grateful to Prof. Lynn Andrea Stein (MIT-LCS) for guiding me towards

JavaSpaces and also to Ligia Cabrera, Sandra Ferolito, Steven Rohall, and Steve Foley (all from Lotus Research) for their help along the way. Additionally, I would like to say thank you to my parents for their love and understanding and their motivation to see me succeed.

Contents

2.2 DSpace and ServerSpace . . . . 25

2.2.1 Commands . . . . 26

2.2.2 Reading and Writing . . . . 27

3 Design Considerations

3.1 Polling versus Notification . . . 3.2 Using Java ObjectStreams . . . 3.3 Server-side Code Execution . .

3.4 Broadcast versus Multicast . . .

4 Implementation Details 4.1 Notification Types . . . . 4.2 Tools . . . . 4.2.1 Tim er . . . . 4.2.2 Cache . . . . 4.3 Managing Connections . . . . . 4.3.1 SametimeOutputStream 4.3.2 SametimeInputStream 4.3.3 Connection . . . . 4.4 The Bot . . . . 4.4.1 ConnectionThread . . . 4.4.2 Connections . . . . 4.4.3 ServerSpace . . . . 4.4.4 Servers . . . . 4.4.5 Place . . . . 4.4.6 Places . . . . 4.4.7 NotifTypes . . . . 4.4.8 NotificationThread . . . 4.5 The Client-Side . . . . 4.5.1 Applet Lifespans . . . . 4.5.2 DSClient . . . . 4.5.3 DSpace . . . . 4.5.4 SametimeSession . 4.5.5 GlobalIM . . . . 33 33 34 34 35 37 . . . . 3 7 . . . . 3 9 . . . . 3 9 . . . . 4 0 . . . . 4 1 . . . . 4 1 . . . . 4 2 . . . . 4 2 . . . . 4 3 . . . . 4 3 . . . . 4 4 . . . . 4 4 . . . . 4 6 . . . . 4 6 . . . . 4 7 . . . . 4 7 . . . . 4 8 . . . . 4 8 . . . . 4 9 . . . . 5 0 . . . . 5 0 . . . . 5 1 . . . . 5 1

4.5.6 GlobalPlaces . . . . 52 4.5.7 SametimePlace . . . . 52 4.5.8 APlace. . . . . 52 5 Results 54 5.1 Demos . . . . 54 5.1.1 Chess . . . . 54 5.1.2 M ap . . . . 56 5.1.3 Jigsaw . . . . 57 5.2 Observations. . . . . 60

5.2.1 Issues with Java ObjectStreams . . . . 60

5.2.2 Issues with the Java Security Model . . . . 61

5.2.3 Latecomer Problem . . . . 62 6 Conclusion 66 A Glossary 68 B Code 69 B.1 Package dominospaces . . . . 69 B.2 Package dominospaces.impl . . . . 73 B.3 Package dominospaces.domino . . . . 79 B.4 Package sametime . . . . 83 B.5 Package tools . . . . 93 B.6 Demos . . . 100 B.6.1 Chess . . . 100 B.6.2 Map . . . 103 B.6.3 Jigsaw . . . 109

List of Figures

1-1 JavaSpaces Model... .

1-2 Example: Chess Application . .

2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9 Overview . . . . Client-Side Applets . . . DSpace and ServerSpace DSpace and ServerSpace Reading Data . . . . Reading Data Example . Writing Data . . . . Data Types . . . . Notification . . . .

Detail

3-1 Notification using a place . . . .

4-1 Notification places . . . . 4-2 Connections on the Bot . . . . 4-3 Generating Notifications on the Bot 4-4 DSClient . . . . 5-1 5-2 5-3 5-4 Chess Application . . . . Map Application . . . . Jigsaw Application... Jigsaw Application (Solved)

5-5 Splice Algorithms . . . . 10 17 22 23 25 26 28 29 30 31 32 36 38 46 48 50 . . . . 55 . . . . 56 . . . . 58 . . . . 58 . . . . 5 9

5-6 Caching in Java Object Streams . . . . 61 5-7 The Latecomer Problem . . . . 63

Chapter 1

Introduction

Lotus Domino' is the flag-ship product of Lotus Development Corporation. Domino is mostly a database software package. While Domino provides numerous ways to access and process data in databases, it does not provide a system in which multiple users can collaboratively process shared data with notifications. Users can modify data (much like in a file system); however, there is no mechanism by which the actions of one user are immediately made known to other users. This feature is known as awareness and if it could be implemented then Domino could support a wide range of applications that involve awareness.

In this thesis, the term "awareness" is used in two ways; awareness of other col-laborating users, and awareness of changes to shared data. Awareness of other users involves each user knowing who else is participating in some mutual collaborative application. These users are typically processing shared data.

Awareness of changes to data involves each user knowing when the data he/she is sharing, has been modified by another user. A user must be notified soon after the data has been changed in order for this user to be considered aware.

To enhance Domino with awareness, an abstraction known as JavaSpaces[4] was explored. This thesis is an attempt to use a model derived from JavaSpaces to provide an architecture for access with awareness, to Lotus Domino. This new model is called

Figure 1-1: JavaSpaces Model

Data Space

Read Notificationa

Write

Client 1 Client 2

"Domino Spaces." The JavaSpaces model is described in the next section.

1.1

JavaSpaces Model

This section describes the JavaSpaces abstraction that was used to develop Domino Spaces. Figure 1-1 gives an overview of this model. JavaSpaces introduces the notion of a "Data Space." Clients wishing to collaborate connect with a data space and are able to read and write data located in that space. A client may inform the JavaSpaces server that it is interested in some data type. Afterwards, the server will inform that client whenever data of the specified type is written to the data space. At this point the client could take some action depending on the data, for example, updating its local cached copy.

JavaSpaces allows clients to process shared data in two ways - synchronously and asynchronously. Synchronous collaboration involves users being aware that other users are modifying mutual data. This awareness allows each user to know when the data he/she is interested in, is modified by some other user. Each user's client machine can then perform some action depending on the kind of modification performed (eg:

each client may update its local copy of the data to reflect the change). The important thing about synchronous collaboration is that changes to data are propagated, as soon as they happen, to all the clients involved.

In contrast, asynchronous collaboration does not require the users to be aware of changes to mutual data. Users may read and write data with no knowledge of what has happened to the data between a read and the corresponding write. In some implementations a user may be warned about changes to sensitive data just before that user attempts to overwrite that data.

Most collaborative systems either offer synchronous or asynchronous collabora-tion, but usually not both. Systems that do provide both options do not have a simple mechanism for switching between the two. Switching to synchronous mode usually involves inviting other users to participate in a virtual chat (or meeting). In some cases the collaborators would have to wait until each of the invitees acknowl-edges or does something pro-active. It would be nice to have a system in which switching between the two collaborative modes occurs with little cost, where cost is measured in terms of user interaction and time.

The JavaSpaces model facilitates the smooth transition between the two collab-oration modes. Synchronous collabcollab-oration takes place when a user registers his/her interests with the server. The server then keeps that user notified and updated. At any point a user may switch back to asynchronous mode by cancelling his/her in-terests with the server. The server will then no longer send notifications to that user. The cost associated with registering and unregistering interests with the server must, of course, be low. Domino Spaces inherited this seamless transition between synchronous and asynchronous collaboration.

The JavaSpaces model was only a starting point. It lacked certain features, and parts of its specification were inapplicable. Therefore, an extended version of this model, which solves these inadequacies, was used to construct Domino Spaces. The shortcomings of JavaSpaces are that JavaSpaces does not provide any awareness of other users, JavaSpaces does not deal with persistent data, and the notification scheme used by JavaSpaces is computationally too expensive. These shortcomings are

de-scribed in more detail next.

Awareness of Other Users

The JavaSpaces model was extended to provide users with knowledge of who their collaborating partners are. In the JavaSpaces model, clients do not have identities. Therefore, there was no awareness of other users incorporated into the model. Domino Spaces extends this model with this particular awareness. In addition to notifications about changes to data, the Domino Spaces server sends notifications about which clients are also working with the same types of data. The idea is that users interested in the same data type are interested in the same data and might want to be aware of one another.

Persistence of Data

The JavaSpaces model deals with volatile data. Each piece of data has a specific time to live. When this time expires, the data is lost. Therefore, there is no delete command built in to JavaSpaces. In addition, multiple copies of the same data could be written to the server.

In contrast, Domino Spaces deals with persistent data. Data that is written, ex-ists until it is over-written or deleted. A command that performs a delete is the only way to remove an object from the persistent store. It is not possible to have multiple copies of an object, each copy would over-write the previous copy. Therefore, the JavaSpaces commands needed to be modified to better suit the needs of a persistent data space.

Cheaper Notification Scheme

Implementing notification in the JavaSpaces interface is computationally intensive. The type of a notification request is determined by examining the public fields of a notification template object. For a notification to be issued, the fields of the object being written must match the fields on the notification template object. Setting a field to null causes that field to behave like a wild-card, matching all values for that

field. This means that for every notification request a vast number of permutations must be considered.

In Domino Spaces, a different strategy for notification was adopted. Each object exports a finite set of notification templates. The idea is that the designer of the object best knows what notifications a typical user of that object will require. Therefore, only those templates are exported, and this greatly reduces the amount of work the server must do.

In JavaSpaces, the object space is flat for the purposes of notification, and this strategy was inherited by Domino Spaces. Because of the hierarchical structure of the server-side storage, objects that are written may modify the states of other objects. However, only one notification (for the object that was directly written) is generated. This solves the problem of having the server "know" which objects depend on which objects. This problem could be quite complex given that it is possible to have mutual dependencies.

1.2

Lotus

This implementation of Domino Spaces made heavy use of software developed by Lotus Development Corp. The two software packages used were Lotus Domino and Lotus Sametime2. Domino can be described as an integrated database and Sametime is a package that offers "chat" capabilities, including instant messaging and services for supporting online meetings. Domino Spaces provided the opportunity to integrate these two packages.

1.2.1

Lotus Domino

Lotus Domino was used as the non-volatile storage on the server-side. Data stored in Domino has a hierarchical index. Every piece of data is indexed by database, document and item.

2

Database A Domino database defines a top level context for a piece of data. Ab-stractly, a database is a collection of documents. Physically, a database is a single file on the underlying file system.

Document A Domino document defines a collection of items. A document rep-resents the smallest quantum of data that can be saved onto the underlying persistent storage.

Item A Domino Item is the smallest addressable piece of data. It is basically a name-value pair. The item name identifies it within a document (although the name does not have to be unique). The Java API exported by Domino allows items to have numbers, strings or special Date/Time objects as values. An Item can also have a vector (with elements of one of the above types) as its value. Items are part of a document and saving an item implies that the corresponding document is written to storage.

1.2.2 Lotus Sametime

Sametime was used to communicate with the clients. Once clients have connected with the Sametime server, they can send data to the server. The server then relays the data to the destination client. Sametime supports the transmission of raw data bytes.

Session To start using Sametime, A Session object must be created. This object represents a client's identity and only one instance of this object may be created per Java virtual machine. This object is used to create the various service classes (see below).

Authentication All the clients must have a username and password to connect with the Sametime server. The Sametime server is charged with authenticating the user.

Message A Sametime Message represents a connection between two clients. A mes-sage has to be created before data can be exchanged between clients. Mesmes-sages

are bidirectional. A client can create multiple messages with the same client. Place A Sametime Place abstraction represents a context for broadcasting data. A

place is uniquely identified by its name; If two clients enter a place with the same name, they will be in the same place. If the place doesn't already exist it is created.

Clients may enter and leave a place at any time. After entering a place, data may be broadcast to the place. Only clients that are present in the place will receive this data. Clients in the same place are notified when other clients enter or leave the place.

CommunityService This object allows clients to connect with Sametime servers

and log in to the system.

InstantMessagingService This object supports instant messaging. It allows Mes-sages (or connections) to be created between two clients allowing data to be exchanged, and handles sending, or receiving data.

WholsHereService This object allows places to be created, and for information to be broadcasted to places. It also provides awareness of other clients in places. To use Sametime, clients must first log in to the Sametime server. Methods to perform an authenticated login are exported by the CommunityService class in the Sametime Java API. This class also has a method that disconnects (logs the client out) from the server.

All Sametime services use an event-listener model to alert clients to the results of

remote procedure calls; none of the methods block. For example, the login method in the CommunityService class returns immediately. The only way for a client to know if a login attempt was successful or not was to register itself as a listener with the service before making a call to the login method. Now a method in the registered listener will be called when it is known that the login was successful (or unsuccessful). To make a connection between two Sametime clients, it is first necessary to convert the other client's name to a unique identifier. This is known as resolving names, and

it is performed by methods in the CommunityService class.

After client names have been resolved into IDs, message objects can be created to communicate with them. Messages are bidirectional and support both textual and binary data. Messages are created (and/or destroyed) by the InstantMessagingService in the Sametime Java API. Once messages are created, data can be sent by means of other methods in this class. By registering itself as a listener to this service, a client can be notified when messages have been created or destroyed and when data is received via a message.

Sametime exports the class WholsHereService for accessing places. This class allows clients to enter places, send data to places and exit places. By registering a listener with this service, a client can be notified (by means of the Sametime event model) when this client has entered or left a place, about other clients entering or leaving a place and of data being broadcast in a place.

1.3

An Example

A typical example that is used to demonstrate the usefulness of the Domino Spaces

architecture is an application for playing Chess. A user wishing to play chess connects to a domino space and request notifications for chess piece types of a particular chess game. At this point the server informs this user about other users who are connected and are also interested in the same data. This user can use this information to become aware of his/her opponent. Figure 1-2 shows an overview of the chess application.

Then the user's client machine would read the current position of the chess board, and display it for the user. If the user makes a move, that move can be written to the server. If the opponent makes a move, that modification will be propagated to this client via notifications. In this way, this user and the opponent will be able to play using a consistent view of the chess board; they can collaborate synchronously.

Note that in order to play chess together, the users did not have to invite each other to the game. When a user is interested in a game, he/she automatically becomes aware of changes to the game and of other users interested in the same game. When

Figure 1-2: Example: Chess Application Client Server begin Interested in game? _Requesqt_Notifications Become aware Reply

Read state of board _ Chess Board

Keep board Notifications_

consistent

Input Perform updates _ _ Write Data_

Leave game?

Cancel notifications--- -Ignore Comman

a user is no longer interested, he/she may leave the game without preventing other users from interacting with the game. In this way users may play and/or leave a game independent of what other users do.

In addition to JavaSpaces other collaborative systems were studied. The next section describes some of these systems.

1.4

Related Work

This section describes some of the other solutions that exist today for collabora-tion and distributed processing. The systems described are NSTP[1], Thor[3], and CORBA[5].

1.4.1

NSTP

The Notification Service Transfer Protocol (NSTP)[1] is another system developed for synchronous collaboration. Clients connect to an NSTP server and can create, modify or delete objects (or things) on the server. However, the objects are virtual objects and have no persistent state.

Collaboration in NSTP is achieved via "place" constructs. In NSTP a place is a context for synchronous collaboration. Messages sent to places are heard by all clients within a place. If an object within a place is modified, all the clients inside that place receive a notification to that effect. In addition to broadcasting messages to a place, a client may send private messages to a single other client or to a list of clients.

Clients are represented as special objects on the server, so that it is possible to see who else is in a place by viewing the list of objects in the place. Clients can keep track of other clients entering or leaving a place by observing the creation or deletion of the corresponding client-objects.

In Domino Spaces the underlying Sametime Place structure is used to broadcast notifications. This place structure is very similar to the one in NSTP. However, an

NSTP place is a more heavy weight object compared to a Sametime place, since NSTP places also manage the objects that are accessed by the clients.

NSTP objects support a locking mechanism. Clients may lock objects inside

a place and then have exclusive access to the object. Other clients are prevented from over-writing the object until the lock is relinquished. This would have been a useful tool to implement atomicity in Domino Spaces. However, since the underlying persistent store (Lotus Domino) does not support locking of objects, atomicity was

partially achieved by other means.

NSTP places have some special attributes. Places may be locked, preventing other

clients from entering. This might be a useful feature for Domino Spaces for deciding who may participate in a collaboration. However, since the underlying Sametime architecture does not support place locking, it was not incorporated into Domino

Spaces.

1.4.2

Thor

Thor[3] is a system for sharing distributed data across remote clients. In some ways it is similar to the system described here. The objects shared by Thor are persistent, and

may. be distributed ancoss sevral dfiernt serers. Whe-~cn apccsicngr rbj~et, clients

Ci j "%-, A1 U.LL I "L A Uk,'I CXAI,. %JLJ3 o" V %-,.L CkI -&I i'i %-,iu 0%. J V %-x . I',x

download and operate on a cached local copy of the object. The object is written back to the Thor server when the client commits the transaction.

However, Thor does not support notification. A client does not find out about modifications to objects it is interested in, until the client tries to commit the changes, at which point the transaction may fail.

Thor also supports atomic transactions affecting data stored on different Thor servers. Although it is possible to have atomic transactions in Domino Spaces, they can only span one space.

1.4.3

CORBA

CORBA[5] (Common Object Request Broker Architecture) is a system with dis-tributed Objects. Clients process these objects through an ORB (Object Request Broker). The ORB delegates the processing to a registered Object Implementation. In this way the language and implementation of an Object is separated by an ab-straction from the clients. The clients do not need to know how an Object is built; the client accesses methods on the Object by passing a reference to the Object, the method name and a list of arguments to the ORB. The ORB decides which Ob-ject Implementation to use and passes on the request. The reply from the ObOb-ject Implementation is handed back to the client via the ORB.

In CORBA every method call on an Object needs to make a remote request. The request is passed from the ORB to the Object Implementation and back. This could be very inefficient. In Domino Spaces, the clients make method calls directly on

Domino Spaces is similar to CORBA in the way data is abstracted from the intermediary. Although the client needs to use the interface provided by a DSObject, the server does not. The server passes on the DSObject to an implementation (known as a Bot; this is described in section 2.1) similar to how an ORB would pass on a CORBA Object to an Object Implementation.

In CORBA, clients do not support call-back. Only Object Implementations sup-port this so that they can be called-up (by the ORB) when a client needs some operation performed on an Object. Therefore, in order to support notifications in a CORBA system each client would also have to register itself as an Object Implemen-tation so that it may be called-back when a notification is generated.

1.5

Thesis Outline

The next chapter gives an overview of the Domino Spaces architecture. It describes the overall layout and gives a brief description of the four types of commands sup-ported by Domino Space, and on how reading, writing and notification is performed. The following chapter discusses some of the design choices that were made. The next chapter gives a detailed description of how Domino Spaces is implemented. It gives a class by class description of the Java classes that implement Domino Spaces from the bottom up.

The next chapter describes the three demos that were coded to test the Domino Spaces architecture. The demos discussed are applications for playing chess, anno-tating a map and collaboratively solving jigsaw puzzles. The chapter also discusses some of the results. The final chapter is a conclusion.

Chapter 2

Architecture Overview

This section gives an overview of the system. It discusses the overall layout and gives the perspective from the client-side, and Sametime server and introduces a specialized client known as a Bot. It then describes the programmer's abstraction to Domino Spaces and gives the list of Domino Space commands. It goes on to describe how the type of an object is determined and explains how data types are used for reading, writing and requesting/generating notifications.

2.1

Overall Layout

Figure 2-1 gives an overview of the design. In this implementation the clients are Java applets so that they may be downloaded. All the clients connect to a central server. After authenticating the clients, the server connects the clients with a specialized client known as the Bot. The Bot is charged with accessing the Domino database (the persistent store) and executing the requests and the modifications that are required

by the clients. The Bot acts as a proxy between the clients and the data.

Each applet has a Lotus Sametime component embedded in it, allowing the clients to communicate using Sametime. Once the applet has been loaded, it opens a con-nection with the Sametime server. The client must authenticate itself to the server. This can be done by reading the user's name and password, and presenting them to the server.

Figure 2-1: Overview Server Java Applet - HTTP Server Sametime _ _ _ Sametime Client Server Domino Bot Database

Once authentication is successful, a client may send messages to a Bot. The messages are in the form of commands that ask the Bot to read or write data to the database. In the case of a read the message describes what piece of data must be read; the Bot then sends back that data. In the case of a write the message describes the data and includes a value; the Bot then replaces that data with the new value.

The Bot performs an additional critical task. Clients may indicate to the Bot that they are interested in certain types of data. The Bot keeps track of which clients are interested in which data, and whenever a write command is executed, the Bot sends a notification to all the clients listed as being interested in the type of data that was modified.

For this thesis, I was responsible for implementing the client-side Java applets and the Bot. These sections are shown with a bold frame in figure 2-1. The following section describes the client-side.

Figure 2-2: Client-Side Applets

2.1.1

Client-Side Overview

Work on the client-side is split up among several applets. Different applications are implemented as different applets. As the user switches from application to application, it would be a bad design if he/she had to log in again each time for each application. Therefore, another applet (the main applet) is charged with logging in. This applet is embedded in a frame that is always present in the web page, so that the applet would not be destroyed. The application applets use the structures created by this applet to communicate with the server.

Please refer to figure 2-2. In this example Applet1 is charged with maintaining the

connection with the Sametime server. When the base frame is loaded, this applet gets the user's name and password and logs into the Sametime server. Applet1 creates all

the static objects that are shared by the application applets. The base frame remains unchanged while the user is using the applications. Since Applet1 is embedded in this

frame, it continues to run and maintain the connection with the server.

The other applets are application applets. They access different data on different databases, but they all communicate via the shared state created by Applet1. They

can share state since the codebase for all the applets is the same, and therefore, they Web Page

Other frames Base frame HTTP

Applet2 Sametime Application1 Applet1 rvr Applet3 Application2 Bot

-run in the same JVM.

To communicate using Sametime, the main applet (in the above example: Applet,) opens a socket connection with the server. This causes the server to assign the client a unique ID, and a name which is used to identify this client to other clients for the purposes of awareness. Depending on the implementation, the server might force the client to authenticate itself before accepting a connection. The Sametime server's role is described next.

2.1.2 Sametime Server

The Sametime server manages the communication between the clients and the Bots for Domino Spaces. An introduction to the Sametime protocol is given in section 1.2.2. Each client (and Bot) begin a Domino Space session by connecting to the Sametime server. The server has a client database and authenticates each client against the credentials in this database.

Sametime's instant messaging feature was used to make connections between clients and Bots. A client sends command requests to the Bot, and receives reply data from the Bot via the instant message system.

Sametime's place architecture was used to broadcast notifications. The Bot creates a unique place for each type of notification requested. All the clients interested in a type of notification enters the corresponding place and receives the notifications being broadcast by the Bot.

2.1.3 The Bot

The Bot acts as an interface between the clients and the databases. It is a special-ized Sametime client and its primary purpose is to provide notifications since this is what Domino does not support. The Bot receives commands from the clients via the Sametime server. Since the clients use the Bot to read and write data in the databases, the Bot knows what data has changed and can therefore, generate the

Figure 2-3: DSpace and ServerSpace

The next section The programmer has

describes the programmer's abstraction into Domino Spaces. access to data via the DSpace class.

2.2

DSpace and ServerSpace

In this system, the context for performing operations is the database. Each connection is associated with a database, and all read, write and notify operations take place on data in that database. Figure 2-3 gives an overview of the high-level objects that connect a client with a database.

On the client-side, high-level communication with the Bot is performed through the DSpace class. The DSpace object is the programmer's interface into the domino spaces system. This class supports reading and writing of DSObject data, and supports requests for, and cancellation of notifications. Each DSpace object operates on a single remote database. Clients may create multiple DSpace objects to access multiple databases.

Each DSpace communicates with the Bot via a Sametime instant message connec-tion. On the Bot, these service requests are handled by a ServerSpace object. There is

ClientBo

Sametime Server Dmn

Figure 2-4: DSpace and ServerSpace Detail Server 1Domino Spacel Databasel Client2 DSpace Client1i Bot Dtabase2 DSpace2 4

a unique ServerSpace object for each database that is referenced by the clients. Mul-tiple DSpaces may be connected with the same ServerSpace if they all reference the same database. Figure 2-4 demonstrates how multiple clients with access to multiple databases interact with the Bot.

The next section describes the types of commands that can be issued by a client using a DSpaces object. These commands are sent via Sametime to the corresponding

ServerSpace object on the Bot.

2.2.1

Commands

There are four types of commands that a client can use. They are Read, Write, Notify and Ignore, used for reading, writing, requesting notifications and cancelling notifications, respectively.

Read Read is used to read data from the server. It takes, as an argument, a list of

object templates. Each template identifies the object to be read. The server returns a list of objects that are referenced by the templates listed in the read request. Section 2.2.2 gives a description of object templates.

Write This command writes a list of data to the server. It also accepts a maximum

cache time. This is the longest time an object may be held in a memory cache before it is written to the persistent store on the server. Other clients will access the latest copy of an object even if it has not yet been written to the persistent store.

Notify Notify is used to request notifications from the server. This command passes a list of data types to the server. Whenever a client writes data matching one of the types, a notification is sent to the requesting client. In addition, this command causes the server to send awareness notifications about other clients interested in the same data types. This command returns an event registration object that may be used to uniquely identify this request. A client uses this command to work synchronously.

Ignore This command is used to cancel notification requests. A client provides an

event registration object (obtained from a previous Notify command request) as an argument and causes the server to no longer generate notifications identified

by this event registration object to this client. A client uses this command to

cancel synchronous collaboration and work asynchronously.

The following section describes how data is accessed in Domino Spaces. This section introduces the DSObject class.

2.2.2

Reading and Writing

Every object that can be written to or read from the Domino Space server must extend the DSObject class. Therefore, all objects in a Domino Space inherit from this class and every DSObject represents some state in a database. For example, Items and Documents are all instances of DSObject.

To reduce the complexity of the Bot, it was decided that the Bot should not have to know what the exact object is, and how it should be accessed. Instead, every DSObject has methods for reading and writing its state from/to a given Domino

Figure 2-5: Reading Data

Client The Bot

Template Object Read Template object No Data _Read Dtbs method + -Dtbs taata Object Data

Database. The DSObject exports a common interface to the Bot, and it is through this interface that the Bot is able to read and write the object to the database. The advantage of this method is that the system can easily be extended to handle different types of objects without having to modify the Bot.

To do a read, the client must send a template of what is to be read to the Bot (please refer to figure 2-5). This template is actually a DSObject. It has enough information in it so that the data it refers to can be identified. However, it does not have any data as yet.

This template is sent to the Bot. The Bot then calls the appropriate read method in the template with the proper database as an argument, causing the template to read the data (it refers to) from the database. This template DSObject is now considered a state object and is sent back to the requesting client. The client can then query the state DSObject for the data.

Figure 2-6 gives an example of a read request. In this example, a data object of type "Item" is being read. There are two fields that uniquely identify the data

Figure 2-6: Reading Data Example Client Item Template Name: Foo Doc: 123 No Data Item Object Name: Foo Doc: 123 Data The Bot Read Database

referred to by this DSObject. They are a name field and a document ID field. In this case, the data being read is the value of the Domino Item (Domino is described in section 1.2.1) with name "Foo" in the document with ID 123.

The Bot does not use the fields in this template. Instead, the Bot invokes the template's Read method on the appropriate database. The template will then read the correct data from the database. This Item object is now sent back to the client.

Writes are performed in much the same way. Figure 2-7 gives an example. The client sends a DSObject over to the Bot. This object is like the template object discussed previously, but it contains data. The Bot calls the appropriate write method on the object causing its internal state to be written to the database. A suitable reply is sent back to the client.

The following section describes how the type of a data object is computed. In Domino Spaces data types are used to uniquely identify an object and also to identify a whole class of objects.

Item Template Name: Foo Doc: 123 Read method Data

Figure 2-7: Writing Data

Client The Bot

State Object Write State Object Data _Write Database t Data Reply

2.2.3

Data Types and Notification

In this system, data types have two parts. The first part is the Java class name of the data object. The second part is determined by the data object itself (via a method call). Figure 2-8 gives an example. In this example, two Item Objects of class

DSpace.Item are shown. In one object, the name of the item is specified, and this is reflected in its data type DSpace.Item,Doc:123,Name:Foo. The other object

has no particular name, and this is also reflected in its data type.

The first type identifies a unique Item. This is an Item named "Foo" and it is contained in a document with ID 123. The second type identifies a whole group of Items. This group consists of all the Items (regardless of name) contained in the document with ID 123.

This method of computing types allows each object to determine its own type and at the same time limit the scope to just that object's class; that is, an object cannot pretend to be an object in another class.

In Domino Spaces any data that is exchanged between the clients and the Bot

must extend the super class DSObject. This object has a method, getKey(, that returns its data type. This type is a function of the class name of the object and

Figure 2-8: Data Types

Object Class Name Data Type

Item Object

Name: Foo DSpace.Item DSpace.Item,Doc: 123,Name: Foo Doc: 123

Item Object

Name: <null> DSpace Item DSpace.Item,Doc:123 Doc: 123

some internal state of the object.

Why is it necessary for an object to determine its own type? It has to do with

the way notifications are performed.

Clients request notifications by specifying the data types they are interested in; the Bot maintains a table of data types and clients. Figure 2-9 gives an overview of this process. When an object is written to the database, the Bot queries the object for a list of data types that must be notified. For each data type, it looks up the list of clients that have requested notifications of that type. The Bot then compiles and sends notifications to each of those clients. In the example in figure 2-9 the Bot will notify the clients 1,2,3,4,6 and 8. Client2 will receive two notifications, one for each data type.

Why were notifications for two data types issued although only one object was

written? A client may want to be notified when an item with a particular name in a particular document is modified. In the above example, this corresponds to the data type DSpace.Item,Doc:123,Name:Foo. The name of the item is "Foo" and it is located in document with ID "123."

Figure 2-9: Notification

is modified. This corresponds to the data type DSpace.Item,Doc:123. Therefore, in the example in figure 2-9, when the Item Object was written, two classes of clients needed to be notified; those clients that were interested in that particular item, and those that were interested in any item change in the document that the item resides. The only reason the client can make such requests is because the Item Object exports such requests. The creator of the Item Object would have had an idea of how the object would be used, and therefore, would export those capabilities. This system gives each object the flexibility to support a wide variety of notification options that are specific to that object, and at the same time does not require the Bot to know the details. This reduces the complexity of the Bot.

DSpace. Item Write Object DSaeIe

Name: Foo Notify which DSpace.Item,Doc:123 Doc: 123 data types? DSpace.Item,Doc: 123,Name: Foo

Lookup Table

DSpace.ItemDoc:123 Clienti, Client2, Client3 DSpace. Item,Doc:123,Name:Foo Client4, Client6, Client8, Client 2

Chapter 3

Design Considerations

This chapter describes some of the design choices that were made in this implerenta-tion of Domino Spaces. It discusses polling versus notificaimplerenta-tion, the use of Java Object streams, server-side code execution, and broadcast versus multicast.

3.1

Polling versus Notification

One way to implement collaboration would be to have the clients regularly ask the server for any changes to sensitive data. In this system the server would not have to keep track of who is interested in what. However, this model has a few significant disadvantages. The network traffic in a polling model increases linearly with the number of clients regardless of the amount of data being shared. This means that even though very little data is being shared, there is a huge amount of network traffic because every client is polling regularly.

Even though lots of data might be shared, the frequency of "writes" to the data might be low. The polling model does not take advantage of this fact to reduce network traffic. A notification system performs better since notifications are sent out only when a "write" is performed.

Finally, the notification system alerts a client relatively soon after changes have been made to the shared data. Conversely, in a polling system, the time for changes to propagate across all clients depends on the polling frequency of individual clients.

After considering all these issues, it was decided to proceed with a notification system. This also provided the opportunity to make use of the broadcast facilities provided by Lotus Sametime.

3.2

Using Java ObjectStreams

Early in the design phase, it was decided to use Java ObjectStreams to send data across the connections. This was to reduce programming overhead; objects could be written to the stream, and it would take care of serializing the object, sending its state across, and reconstructing it at the other end. However, the serializing technique used

by the default Java ObjectStream is terrible. It adds on a lot of overhead in terms of

data size. It was observed that approximately 500 bytes of overhead were sent with

100 bytes of data (Please refer to section 5.2.1).

Therefore, a more practical design would not use the Java ObjectStreams and perform a more efficient serialization. This would mean that each DSObject would include code that would serialize and deserialize its state.

3.3

Server-side Code Execution

Since this implementation has a back-end permanent store (the Domino Database), the server needs to know how to read/write objects from the store. Instead of hard-coding the methods for performing these file accesses into the server, these methods are included with the DSObject being written. That is, each DSObject has its own specific methods for reading and writing its state to and from the store.

This allows custom objects to be written on the client-side, and so enhances the power of the client-side API. However, this also means that potentially hazardous code could be made to execute on the server-side. The correct solution depends on the way the server will be used and so depends on the application. This architecture offers a lot of flexibility, but when it is used, it will be made more restrictive, depending on the specific application. For example, the server could be forced to load the class files

from the local file system. Then any objects that a client uses should have its class files already on the server. Only trusted objects will have their class files residing on the server. This is one way of controlling the code that executes on the server-side.

3.4

Broadcast versus Multicast

These are design decisions considered for delivering notifications to clients. Broad-casting is a system where the server keeps a list of clients ordered under groups. A client may broadcast a message to a number of users by addressing the message to a group. The server then sends a copy of the message to each of the clients listed in the group.

Multicasting involves the client sending a message to the server with a list of destination clients. The server then sends a copy of the message to each of the clients listed in the destination list.

The Sametime software does not support multicasting. Therefore, it was neces-sary to use the broadcast mechanism to propagate notifications. However, the only broadcast mechanism supported by Sametime is broadcasting via Places. This could be inefficient because the Place architecture sends additional notifications about who has entered, and/or left a Place.

This potential inefficiency was actually taken advantage of, in this implementa-tion. The extra notifications were used to provide awareness of other clients interested in the same information as a given client.

Notification Using Sametime Places

To use broadcasting, a virtual place must be created, and all the clients wishing to participate must enter the place. If any client writes a message to the place, that message is received by all the other clients who are also in the same place.

Domino Spaces made use of this architecture to perform notifications. The Bot creates unique places for specific notifications. If a client is interested in a notification type, it will ask the Bot for the unique place name and enter that place. The client

Figure 3-1: Notification using a place Sametime Place

will then begin to receive notifications of that type. When the client is no longer interested, it may simply leave the place to stop receiving notifications. Please refer to figure 3-1.

Places are created only if clients request a specific type of notification and there is currently no place created for that type. When all the clients in a given place leave

(no one is interested in that type anymore), the place is destroyed.

Lotus Sametime also generates events when clients enter or leave a place. This was incorporated into Domino Spaces to provide awareness of who is interested in some specific data type.

Broadcast

Chapter 4

Implementation Details

This chapter explains notification by type in more detail. It then discusses the Java classes that were used to build this implementation of Domino Spaces. A bottom up strategy is used to present these classes. First, some general purpose classes are described. Then some classes that facilitate working with Lotus Sametime, and finally classes that are specific to Domino Spaces are described.

4.1

Notification Types

There is a subtle difference between data types and notification types.

Notification Type =

f(Data

In other words, a notification type is a function of several data types. A single notification type defines a list of data types. A client may request notifications for several data types, in which case it signs up for a single notification type and enters a single place. This makes the server more efficient in terms of memory space as places may be heavy weight objects.

The lookup table in figure 2-9 must be changed. The updated lookup table is shown in figure 4-1. The table now maps types to places. There is a unique place for each notification type. Clients wishing to receive specific notifications enter each re-spective place. In the example in figure 4-1, Client₂has signed up for two notifications

Figure 4-1: Notification places

types.

Places may receive notifications for multiple data types. In the above example,

Place3 receives two types of data types. Client9 receives both these types. Client9 and

Client2 receive the same types of data, although they have entered different places.

The reason for this is that Client2 has signed up for two different notification types,

whereas, Client9 has signed up for one notification type which is a function of two

data types.

From the server's point of view it might be more efficient to use the method used

by Client9 as it creates one place instead of two. However, Place1 and Place2 might

also be needed for clients who want only one type of notification, in which case it is more efficient to use the method used by Client2 as this will minimize the number of

places created. In this implementation the choice of which method to use is left to

the developer.

Bot

Placel Place2 Place3

Notifications Notifications Notifications

of Typel of Type2 of Type3

Clienti Client2 Client4 Client9

Lookup Table

DSpace.Item,Doc: 123 Placel Place3 DSpace.Item,Doc:123,Name:Foo Place2 Place3

4.2

Tools

The classes in this section are general purpose tools that are unrelated to Domino Spaces, but are used in this implementation. The two utility classes are the Timer and the Cache.

4.2.1

Timer

It was necessary to have certain tasks occur after a certain time delay. Java 1.3 (with its Timer class) had not come out yet, and so a class to schedule events had to be developed (It was naturally called Timer). This class ran in its own thread. Events (with an associated delay time) could be added to the Timer's queue. These events all implemented the Runnable interface, and after the respective delay time had expired, their run methods were called. The run methods executed in this classes thread, so it was necessary that these methods execute quickly. After an event was executed, it was removed from the queue.

This Timer was implemented using a binary heap structure with a heap property of each parent node having lower cost than its child nodes. In this case, the cost was the execution time, each node represented a task, and the task that needs to be executed next was at the root of the tree. This heap structure allowed additions and deletions of tasks (nodes) in O[Log(n)] time, where n is the number of tasks already scheduled. This is a very efficient time for this kind of operation.

The Timer thread might have to sleep, depending on when the next task must execute. It could be inefficient to sleep for small periods of time, so the Timer class has a minimum-sleepitime setting. If the Timer thread decides to sleep, it will sleep for at least the time in this setting. This means that some tasks could get executed at a later time compared to their requested delays. The longest time a task could sit in the queue before being executed is delay + minimum-sleepitime, where delay is the associated delay time for this task. Once a thread wakes up, if multiple tasks need to be executed, they will be executed in sequence and the thread will not sleep in between executions.

4.2.2

Cache

Very often clients need to write updates to the same data object multiple times over a short period of time. This happens if, for example, a user drags an object across the screen, and the client has to send updates of the location of that object to the data space. This involves the client performing multiple write operations on the same domino space object on the server. It would be inefficient if the system kept writing these changes to the persistent store each time for every write request. What was needed was for the system to perform the writes on a temporary object in memory and commit the object to the persistent store only after a specific period of time. This allows the system to absorb many write requests before actually writing the modifications to the disk.

To implement this efficiently a class that performs caching was developed. When accessing data, the cache is searched first, if the data is in the cache, the modifications are performed on that data object. If it is not in the cache, then it is read from the persistent store and inserted into the cache.

The Cache inherently does read caching although it is not as critical as write caching. Read caching is already performed by the underlying Domino database architecture. That is, if an object is being read, the back-end software first checks to see if the object is already in memory before fetching it from the disk.

The cache was implemented using a hashtable. Objects, with a given key, are inserted into a cache by calling the cache's putO method. They are fetched by calling the cache's get( method using the key as an argument.

Objects that are placed into the cache need to implement the Cacheable interface. This has a isDirtyO method that is called to find out if an object had been modified while residing in the cache. Such an object needs to be written to the disk. Therefore, the interface also has a commit() method that is called when an object needs to write itself to the persistent store.

Caches have a finite size (specified at the time of creation) and a least recently used (LRU) replacement policy. This was implemented using a linked-list structure. Every

time an element of the cache is accessed, it is moved to the front of the list. When an element needs to be inserted and the cache is full, the element at the end of the linked-list (the least recently used) is evicted to make room for the new element. The linked-list was implemented using both a head and tail pointer effectively providing access to the first and last element in the list.

The data in the cache is, therefore, accessed by two separate indexes. The hashtable is used to access via a key. The linked-list is used to access the least recently used.

The following section describes some of the utility classes that were used in con-junction with the Sametime API.

4.3

Managing Connections

Connections are made between clients and the Bot in order to exchange data. The state of a connection is captured by a Connection object. To manage connections it was necessary to create Java streams for input and output. These streams buffer data bytes and make calls to the underlying Sametime API. The Connection object manages the two streams that communicate on this connection.

4.3.1 SametimeOutputStream

This object implements a Java OutputStream to which data that needs to be sent across the connection can be written. This object makes calls to the underlying Sametime API to actually send the data. To send data by Sametime it is necessary to provide both a data type and a data subtype. Therefore, this object has a method that allows these types to be set. The default types are zero for both.

Data can be sent using Sametime in two ways, either by calling the sendData method in the InstantMessagingService, or by calling a similar method in the WhoIsHereService. For this OutputStream to work for both services, an interface was created that would provide a common abstraction into both these services. This interface is called SendDataInterface.

To send data, an ID must be provided. If the InstantMessagingService was used, then this ID would be the message ID. Otherwise, it would be the place ID.

SametimeOutputStreams are used by the Bot to send data to a single client and also to broadcast data to a place. On the receiving end of the stream, the class SametimeInputStream is used to read bytes written to a SametimeOutputStream.

4.3.2 SametimelnputStream

This class is responsible for reading bytes generated by the corresponding Same-timeOutputStream. It extends the Java InputStream class and so supports all the methods required by InputStream.

Sametime supports an event driven call back system. When data is received specific methods on registered listener classes are called with appropriate data. Data can be received from two services. Data from an instant message is received by a call back from the InstantMessagingService, while data broadcast to a place is received

by a call back from the WholsHereService class. The SametimeInputStream class

is designed to work with both services. It has a method called dataReceived that must be called by a service listener with the data. The data can then be read from this class using the standard read methods supported by InputStream.

4.3.3 Connection

The Connection object is a data structure that represents a connection between a client and a Bot. It keeps track of the SametimeInputStream and SametimeOutput-Stream that is used to communicate through the connection. The InputSametimeOutput-Stream is used to read data from the connection, and the OutputStream is used to write data to the connection.

The Connection object also keeps track of which Sametime user the connection is connected with. It also has a method to set its data type. The corresponding Same-timeOutputStream will send data only as this type, and only data that is received and that has this type will be propagated to the corresponding SametimeInputStream.