This deployment scenario spreads components over multiple servers for the purpose of load-balancing, implicitly including a level of high-availability.
Each functional component in Kolab Groupware can be scaled up, and scaled down, independent from the other components.
A single server deployment provides the following services:
These services can be spread out over multiple systems:
Alternatively, some services can run on one server, while other services run on another:
Services can also be spread out even more, and duplicated for extra processing power, load-balacing, redundancy and/or high-availability:
While the Introduction chapter in the Architecture & Design document included a high-level overview of functional components, in this chapter we can zoom in a little and define the functional components by a role. Generically speaking this would include the following roles:
LDAP
See also
Web services
See also
Data Storage Layer
Databases
Mail Exchangers
See also
To illustrate how this could look like we put it into a diagram:
In a minimally distributed deployment topology, all components speak to a single LDAP server.
This can be extended by replicating LDAP with the introduction of one or more replication slaves dedicated to read-only operations:
And extended yet even further by the introduction of (caching) proxies:
A single web server can not handle the requests of too many users, and even if it could, it would not be high-available. When web services are distributed, especially where it concerns a single web application being distributed, things such as sessions and also some caches must be stored outside the web server node.
This is to prevent a loss of session validity when users turn (out) to hit another web server node for subsequent requests as part of their existing session, and should be taken in to account even when “sticky” load-balancing is employed.
Since this session storage must be available to all web server nodes participating in serving an application, it makes sense to provide this session storage over the network.
To do so in a SQL database however tends to put an unfair burden on the SQL infrastructure, especially when replication is involved.
You could simply use memcached for the session storage, but this moves the single point of failure just one step further – namely should the memcached server, fail, sessions are lost.
To reduce the number of sessions lost, or to increase storage capacity horizontally, one can supply multiple memcached servers to the session storage driver, which will be used as a pool – the sessions would be spread throughout the pool. Loss of a single memcached server is then limits to only a subset of the sessions.
A much improved scenario is assisted by replicated memcached. Pairs of memcached servers replicate with one another so that either can fail.
Backend Mail Exchanger
A backend mail exchanger is also a Cyrus IMAP backend.
Internal mail exchangers use LDAP to look up which Cyrus IMAP backend server system the user’s mailbox or shared folder resides on. They then relay the message to the backend mail exchanger for final delivery, using SMTP.
Kolab Groupware uses SMTP by default, while using networked LMTP is another option.
Todo
We probably need to explain why we choose to use SMTP rather than LMTP.
State something about pre-authorizing LMTP, it happening over the network, murder topologies and LMTP proxying, the use of LDAP holding the mailHost attribute, accounting, etc.
Internal Mail Exchanger
Responsible for internal routing of email, including;
- Relay outbound messages to the external mail exchanger(s),
- Relay inbound messages to the appropriate backend mail exchanger(s),
- Relay messages to appropriate third party groupware or services,
- Expansion of distribution groups into its individual member recipient addresses,
Optionally responsible for the application of anti-virus.
Note
It is probably the responsibility of the external mail exchanger(s) to scan for spam messages. The internal mail exchanger(s) receive mail from the internal network, and from the external mail exchanger(s).
External Mail Exchanger
- Responsible for external mail routing
The following diagram illustrates the split of roles for mail exchangers, and their involvement in a deployment.
In this deployment scenario, each of the roles can be fulfilled by one or more servers. To illustrate, here’s a diagram for a deployment scenario that expects to be hammered by probably illegitimate or unsollicited mail, and does not expect that much traffic internally:
The responsibilities of an external mail exchanger typically include at least:
In addition, the external mail exchanger may be made responsible for an initial check on whether the intended recipient is a valid local recipient. We recommend considering, however:
An external mail exchanger is typically positioned in a perimeter network.
Checking the validity of recipient addresses in the perimeter network implies a (fractional) copy of the LDAP directory tree is available to the external mail exchanger.
Allowing nodes in a perimiter network to directly query internal LDAP servers is an attack vector, and you should thus consider creating a slave replica LDAP server in the perimeter network.
You should consider making this replica a purposeful fractional replica, as to make available in the perimeter network the absolute least amount of information necessary for the external mail exchanger.
Use different bind credentials on the external mail exchanger, and make sure no internal service accounts are replicated.
You should, subsequently, also consider removing unnecessary indexes from the LDAP databases, and reduce the indexes for recipient mail address attributes (mail, for example, has an index for presence, substring presence and equality, with the substring presence being unnecessary).
Aside from a number of appliances to this purpose, one can implement load-balancing with implied high-availability if the Kolab Groupware functional components or roles are each installed on at least two systems.
Multiple functional components may be installed on a single system.
Caution
Most appliances, like this solution based on keepalived, will need the clients connected on failover or failback to reconnect to the new system to which the service IP address in use is attached.
In a normal situation, Server #1 holds the .3 service IP address, while Server #2 holds the .4 service IP address.
DNS for example.org has been configured to hold two IN A records for ldap.example.org:
The client system requests the IN A for ldap.example.org:
The client connects to one (semi-randomly selected) of the resulting IP addresses:
Which server this connection ends up with depends on the health of Server #1 and #2.
In a normal situation:
and:
Should Server #1 fail (its local LDAP service health check(s)):
and: