1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
|
# Rough draft for the intended server architecture
Very messy, just here to get my thought down somewhere.
Each thread is represented as its own 'file' (the exact format used to store
messages is not really relevant, but file-based is easy to handle at least), and
clients 'sync' this file to themselves. Whether a client wants to keep a copy of
the file or always load a new version depends on the client itself and what type
of thread it is. For example, public channels with huge amounts of messages make
more sense to just sync the last 100 messages of at a time.
An account always connects to a 'home server' of sorts, so user@server.com or
whatever. The server takes care of connecting to other servers when need be.
Each account additionally has an event queue, such as "you got invited to thread
ABC by someone@somewhere" which always accepts new events. The client should
request the event queue upon connecting (or should it be done automatically?)
A client tells the home server which threads it's interested in at the moment
via the SYNC command. If the SYNC command is to a thread on another server, a
new connection is opened up (if one doesn't exist already), and the SYNC is
forwarded.
SYNC can take parameters for thread history, so
SYNC 17\nSTART = 10\nEND = -20
skips the first ten bytes and the last twenty (off by one?). The number after
SYNC specifies how many bytes are in the header, i.e. how many characters follow
the first newline.
Omitting the END parameter means that the client is interested in getting new
messages as they come in, i.e. should be placed in a 'listening' state.
The other server should send the home server an update whenever
anyone adds anything to the thread. The home server is responsible for keeping
track of which accounts are interested in which threads, the other server should
only see 'this server is interested in this thread'. If no accounts are
interested in a thread, the home server should send a command to signify this
(CLOSE? STOP? DESYNC? Some other command name?)
Messages are built of a header, consisting of some <STRING> = <STRING>
parameters, and a body that's been encoded with a specific 'lark formatting',
just replacing all = -characters with \= (or %=? Some other combination?).
Clients must take care of encoding and decoding, servers are required to check
that the body follows the ecoding but are not required to fix it for the user.
(I did some benchmarks and checking is significantly faster than fixing)
The server (/collection of server instances?) maintain a shared state, such that
each connection (client <-> server and server <-> server) has its own queue.
When a message is added to a thread, all connections interested in that thread
should have the message added to their outgoing queue. The sender must also be
interested in a thread. Each thread can have limitations on who is allowed to
send to it (or do other operations), which I haven't really thought about yet,
but they should presumably be checked at sendtime as well.
Connection queues are presumably fairly simple, just add files to a directory
and a reference to those files in some queue file that's locked by whoever is
dealing with it. Server <-> server connections are presumably fairly easy as
well, keep a map of threads on the other server that we're interested in and
what account is interested in it, and then just directly add incoming messages
to client connection queues. Messages should be delivered so that at least ones
to the same thread are sent to clients in the order that they arrived, between
threads they can arrive in different orders.
Have to think about how errors are
reported, give each command a UUID or just specify that client should wait for a
response to a command before sending another one? I guess it depends on the
command, SYNC contains enough metainformation to just say "right, this thread
didn't like that" etc.
Server <-> server it's pretty easy to keep track of which queue a message should
be added to, but it's not entirely obvious how the list of which connections are
listening to which thread should be handled. One option could be inotify of the
thread file, but that requires separate handling of inotify/kqueue for
Linux/BSD. Simplest option would arguably just be that each connection is given
a turn to run often enough that a small delay is insignificant, and each
connection queries the relevant local threads on their own, and send out any new
bytes since the thread file was last read. One option I thought about was
creating a named pipe for each thread, and on new messages, the sending
connection handler writes something into it, and listeners poll() for the read
end. Unsure of the semantics here, docs don't outright say something like it
wouldn't work but I'm not sure which all polls() get notified. Also, one FD per
thread, might be a lot for server <-> server connections?
|
