add architecture document

1 files changed, 83 insertions, 0 deletions

diff --git a/architecture.md b/architecture.md
new file mode 100644
index 0000000..104d40f
--- /dev/null
+++ b/architecture.md
@@ -0,0 +1,83 @@
+# Rough draft for the intended server architecture
+
+Very messy, just here to get my thought down somewhere.
+
+Each thread is represented as its own 'file' (the exact format used to store
+messages is not really relevant, but file-based is easy to handle at least), and
+clients 'sync' this file to themselves. Whether a client wants to keep a copy of
+the file or always load a new version depends on the client itself and what type
+of thread it is. For example, public channels with huge amounts of messages make
+more sense to just sync the last 100 messages of at a time.
+
+An account always connects to a 'home server' of sorts, so user@server.com or
+whatever. The server takes care of connecting to other servers when need be.
+
+Each account additionally has an event queue, such as "you got invited to thread
+ABC by someone@somewhere" which always accepts new events. The client should
+request the event queue upon connecting (or should it be done automatically?)
+
+A client tells the home server which threads it's interested in at the moment
+via the SYNC command. If the SYNC command is to a thread on another server, a
+new connection is opened up (if one doesn't exist already), and the SYNC is
+forwarded.
+
+SYNC can take parameters for thread history, so
+
+	SYNC 17\nSTART = 10\nEND = -20
+
+skips the first ten bytes and the last twenty (off by one?). The number after
+SYNC specifies how many bytes are in the header, i.e. how many characters follow
+the first newline.
+
+Omitting the END parameter means that the client is interested in getting new
+messages as they come in, i.e. should be placed in a 'listening' state.
+The other server should send the home server an update whenever
+anyone adds anything to the thread. The home server is responsible for keeping
+track of which accounts are interested in which threads, the other server should
+only see 'this server is interested in this thread'. If no accounts are
+interested in a thread, the home server should send a command to signify this
+(CLOSE? STOP? DESYNC? Some other command name?)
+
+Messages are built of a header, consisting of some <STRING> = <STRING>
+parameters, and a body that's been encoded with a specific 'lark formatting',
+just replacing all = -characters with \= (or %=? Some other combination?).
+Clients must take care of encoding and decoding, servers are required to check
+that the body follows the ecoding but are not required to fix it for the user.
+(I did some benchmarks and checking is significantly faster than fixing) 
+
+The server (/collection of server instances?) maintain a shared state, such that
+each connection (client <-> server and server <-> server) has its own queue.
+When a message is added to a thread, all connections interested in that thread
+should have the message added to their outgoing queue. The sender must also be
+interested in a thread. Each thread can have limitations on who is allowed to
+send to it (or do other operations), which I haven't really thought about yet,
+but they should presumably be checked at sendtime as well.
+
+Connection queues are presumably fairly simple, just add files to a directory
+and a reference to those files in some queue file that's locked by whoever is
+dealing with it. Server <-> server connections are presumably fairly easy as
+well, keep a map of threads on the other server that we're interested in and
+what account is interested in it, and then just directly add incoming messages
+to client connection queues. Messages should be delivered so that at least ones
+to the same thread are sent to clients in the order that they arrived, between
+threads they can arrive in different orders.
+
+Have to think about how errors are
+reported, give each command a UUID or just specify that client should wait for a
+response to a command before sending another one? I guess it depends on the
+command, SYNC contains enough metainformation to just say "right, this thread
+didn't like that" etc.
+
+Server <-> server it's pretty easy to keep track of which queue a message should
+be added to, but it's not entirely obvious how the list of which connections are
+listening to which thread should be handled. One option could be inotify of the
+thread file, but that requires separate handling of inotify/kqueue for
+Linux/BSD. Simplest option would arguably just be that each connection is given
+a turn to run often enough that a small delay is insignificant, and each
+connection queries the relevant local threads on their own, and send out any new
+bytes since the thread file was last read. One option I thought about was
+creating a named pipe for each thread, and on new messages, the sending
+connection handler writes something into it, and listeners poll() for the read
+end. Unsure of the semantics here, docs don't outright say something like it
+wouldn't work but I'm not sure which all polls() get notified. Also, one FD per
+thread, might be a lot for server <-> server connections?