April 17, 2011

 Using Unison with Atmail

Unison allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other. In other words, it can be used for two-way synchronization between your servers.

For illustrative purposes in this example, the Unison Master hostname will be called "master-foo-01"; the Client hostname will be called "slave-foo-02".

To setup Unison on your machine, do the following for both machines:

1.) Download the dependency, OCaml from: http://caml.inria.fr/download.en.html

atmail@master-foo-01# wget "http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.0.tar.gz"

2.) Install emacs (a dependency):

atmail@master-foo-01# yum install emacs

3.) Unpack, install OCaml:

atmail@master-foo-01# tar xvfz ocaml-3.12.0.tar.gz
atmail@master-foo-01# cd ocaml-3.12.0
atmail@master-foo-01# ./configure && make world
atmail@master-foo-01# make opt
atmail@master-foo-01# make install

4.) Download Unison from: http://www.cis.upenn.edu/~bcpierce/unison/download.html

atmail@master-foo-01# wget "http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.63.tar.gz"

5.) Untar, install:

atmail@master-foo-01# make world opt
atmail@master-foo-01# sudo make install

6.) Move the Unison binary to your preferred prefix. We suggest /usr/bin/:

atmail@master-foo-01# mv unison /usr/bin/unison
atmail@master-foo-01# chmod 755 /usr/bin/unison

7.) You will then need to set the keys. Create a SSH public key on the Master server:

atmail@master-foo-01# ssh-keygen -t rsa

8.) This will produce a public key in /home/atmail/.ssh/id_rsa.pub. Copy the .pub file to the Slave machine:

atmail@master-foo-01# scp /home/atmail/.ssh/id_rsa.pub root@slave-foo-02:/home/atmail/.ssh/id_rsa.pub

9.) Pipe the public key into a file called authorized_keys, in your .ssh directory.

atmail@slave-foo-02# cat /home/atmail/.ssh/id_rsa.pub >> authorized_keys

10.) Make sure that the authorized_keys file, and all of the contents of the .ssh directory have 600 permissions:

atmail@slave-foo-02# chmod 600 /home/atmail/.ssh/*

11.) Try to login, sans password, from the Master machine to the Client machine.

atmail@master-foo-01# ssh atmail@slave-foo-02

12.) This should allow you to login. Now, you can try to sync the differences between the users/ directory of the master and slave machines. From any of the machines, execute:

atmail@master-foo-01# unison -batch -auto /usr/local/atmail/users ssh://slave-foo-01//usr/local/atmail/users

13.) The output should be similar to:

UNISON 2.40.63 finished propagating changes at 01:25:25.57 on 18 Apr 2011
Saving synchronizer state
Synchronization complete at 01:25:25  (xx items transferred, 0 skipped, 0 failed)

14.) You will need to set this in cron. Create a file called /home/atmail/unison.sh. In the file:

#!/bin/sh
/usr/bin/unison -batch -auto /usr/local/atmail/users ssh://slave-foo-01//usr/local/atmail/users

15.) Set permissions:

% chmod 755 /home/atmail/unison.sh

16.) Add an entry to your /etc/crontab. It will look like:

01,10,20,30,40,50 * * * * atmail /home/atmail/unison.sh

This will synchronize your /usr/local/atmail/users directory for both machines.


Filed under: Uncategorized, Multiserver, OS, Atmail 5, Atmail 6 — John Contad @ 10:13 pm

 

May 24, 2010

 MySQL NDB for Atmail

This portion of the guide covers the steps necessary to use the MySQL NDB Cluster with two storage engines and one management server. This will allow the two MySQL servers to share data independently, and still have the same information for both. In the case of one server going down, the other server will still be active. Changes made to the other server during this period will be "healed" by the MySQL NDB management system.

Packages for MySQL NDB exist for a lot of systems. However, this guide will cover the process of installing it from source - along with PHP.

For this guide, we will have three servers - server1.local (192.168.0.1), server2.local (192.168.0.2), and server3.local (192.168.0.3). The first two servers will be the replicated storage engines. The third server will act as an NDB management machine.

Setting up the Storage Machines

This step applies to server1.local and server2.local. First, download the MySQL Cluster package from: http://dev.mysql.com/downloads/cluster/. Then, untar to your preferred base directory. For this example, we will use /usr/local/:

% tar xvfz mysql-cluster-gpl-7.1.3-linux-i686-glibc23.tar.gz -C /usr/local/

Create a symlink for ease, or rename the folder:

% ln -s /usr/local/mysql-cluster-gpl-7.1.3-linux-i686-glibc23 /usr/local/mysql

Then, install MySQL:

% cd /usr/local/mysql
% adduser mysql
% groupadd mysql
% scripts/mysql_install_db --user=mysql
% chown -R root  .
% chown -R mysql data
% chgrp -R mysql .
% cp support-files/mysql.server /etc/rc.d/init.d/
% chmod +x /etc/rc.d/init.d/mysql.server
% chkconfig --add mysql.server


Then, create a file called /etc/my.cnf. In the file, add:

[mysqld]
ndbcluster
ndb-connectstring=192.168.0.3
[mysql_cluster]
ndb-connectstring=192.168.0.3

Replace "192.168.0.3" with your management server's (server3.local) IP. Then, create the required directories:

% mkdir /var/lib/mysql-cluster

Setting up the Management Server

The MySQL Cluster package will contain the following files:

bin/ndb_mgm
bin/ndb_mgmd

Transfer these files to the server3.local server, as:

/usr/bin/ndb_mgm
/usr/bin/ndb_mgmd

Then, assign permissions:

% chmod +x /usr/bin/ndb_mgm
% chmod +x /usr/bin/ndb_mgmd

Then, create the required directories:

% mkdir /var/lib/mysql-cluster

In the directory, setup the configuration file called "/var/lib/mysql-cluster/config.ini". In the file:

[NDBD DEFAULT]
NoOfReplicas=2
[MYSQLD DEFAULT]
[NDB_MGMD DEFAULT]
[TCP DEFAULT]
# NDB MGM Server
[NDB_MGMD]
HostName=192.168.0.3
# MySQL servers
[NDBD]
HostName=192.168.0.1
DataDir= /var/lib/mysql-cluster
[NDBD]
HostName=192.168.0.2    
DataDir=/var/lib/mysql-cluster
[MYSQLD]
[MYSQLD]

Change the IP addresses accordingly, depending on your setup. Now, start the NDB management process:

% /usr/bin/ndb_mgmd

Starting the NDB processes

On the two MySQL servers, execute:

/usr/local/mysql/bin/ndbd --initial
/etc/rc.d/init.d/mysql.server start

You can check if the MySQL NDB is active by typing this in the third server (server3.local):

% ndb_mgm

This will show a terminal. Type:

ndb_mgm> show

It will show something similar to:

Cluster Configuration
---------------------
[ndbd(NDB)]    2 node(s)
id=2    @192.168.0.1  (mysql-5.1.44 ndb-7.1.3, Nodegroup: 0, Master)
id=3    @192.168.0.2  (mysql-5.1.44 ndb-7.1.3, starting, Nodegroup: 0)

[ndb_mgmd(MGM)]    1 node(s)
id=1    @192.168.0.3  (mysql-5.1.44 ndb-7.1.3)

[mysqld(API)]    2 node(s)
id=4    @192.168.0.1  (mysql-5.1.44 ndb-7.1.3)
id=5    @192.168.0.2  (mysql-5.1.44 ndb-7.1.3)


Congratuations! You have MySQL NDB active.


Filed under: Uncategorized, Database, Multiserver, Atmail 6 — John Contad @ 11:25 pm

 

November 10, 2008

 Slow NFS performance - Increase default RPCNFSDCOUNT

If you are running CentOS/Fedora, by default the NFS daemon will only spawn up to 8 processes. If you have a heavily loaded system, with multiple frontends accessing the storage via NFS the performance will be seriously effected after 8 concurrent connections to the NFS store.

It is highly recommended to increase the default RPCNFSDCOUNT setting when the NFS daemon starts.

Under CentOS/Fedora, create the file:

/etc/sysconfig/nfs

Increase the default setting:

RPCNFSDCOUNT=32

32 is a ballmark figure, increase to 64 or 128 for a highly loaded system.

Next, restart the NFS frontend:

/etc/init.d/nfs restart

Then sit back and watch the performance and load improve on all your frontend servers running Atmail!

You can test the write performance via NFS from the command:

ssh root@frontend.system
su atmailcd /usr/local/atmail/userstime dd if=/dev/zero of=/usr/local/atmail/users/testfile  bs=16k count=1000

This will return the performance and transfer rate between a frontend and the NFS server.

16384000 bytes (16 MB) copied, 5.20865 seconds, 3.1 MB/s

For anything below 1 MB/s the NFS server is too heavily loaded and performance will be effected. Check your switch, your using a gigabit ethernet card, and the local disks on the NFS server are optimized.


Filed under: Uncategorized, Multiserver, Optimization — info @ 5:14 pm

 

January 1, 2008

 Optimizing and NFS performance

If you are using NFS with maildir for storage of messages in @Mail - Common under a Multi-server installation - http://support.atmail.com/multiserver.html

You can run the following commands to check the read/write speed of IO access via the terminal.

# time dd if=/dev/zero of=/usr/local/atmail/users/testfile bs=16k count=16384

This creates a 256Mb file of zeroed bytes. In general, you should create a file that's at least twice as large as the system RAM on the server. The above will time the number of seconds to write the file to test.

To test read access, you can use the following command to read back the file.

# time dd if=/usr/local/atmail/users/testfile of=/dev/null bs=16k

Repeat this a few times and average how long the read/write process takes.

You can also optimize the mount options under NFS, depending on which Linux/Unix OS you use, the defaults can differ greatly.

When mounting the drive you can specify the following options:

# mount -t nfs -o "rsize=8192,wsize=8192" server:/storage /usr/local/atmail/users

Note the "rsize=8192,wsize=8192" flags. This increases the default read/write buffer via NFS. You can also add these changes into the /etc/fstab on boot.

Asynchronous mode can also be enabled using the "async" as an option on the mount flags. It yields better performance at the cost of possible data corruption if the server reboots while still holding unwritten data and/or metadata in its cache/memory.


Filed under: Multiserver — Ben Duncan @ 12:00 pm

 

December 6, 2007

 Multiserver NFS Configuration

f you want the Maildir email message database to be stored on a remote server, the most practical means of doing this is to install it on a remote NFS mount. The webserver can access this mount everytime a user accesses an email; which can distribute the load over multiple servers.

The /usr/local/atmail/tmp/ directory must be shared via NFS. This is required for each frontend to access the single directory storage for attachments and temporary messages.

For this example, we will use the domains webserver.com (as the domain of the webserver) and database.com (for the database server).

(more...)


Filed under: Multiserver — info @ 3:16 pm

 

June 26, 2007

 Multiserver NFS Configuration

If you want the Maildir email message database to be stored on a remote server, the most practical means of doing this is to install it on a remote NFS mount. The webserver can access this mount everytime a user accesses an email; which can distribute the load over multiple servers.

The /usr/local/atmail/tmp/ directory must be shared via NFS. This is required for each frontend to access the single directory storage for attachments and temporary messages.

For this example, we will use the domains webserver.com (as the domain of the webserver) and database.com (for the database server).

First, add the following to the /etc/fstab file of the webserver:

database.com:/usr/local/atmail/users /usr/local/atmail/users nfs rw 0 0
database.com:/usr/local/atmail/tmp /usr/local/atmail/tmp nfs rw 0 0

Then, create an entry in the /etc/exports file of the database-server, which will give read/write permissions for that directory as an NFS mount:

/usr/local/atmail/users webserver.com(rw)
/usr/local/atmail/tmp webserver.com(rw)

Then, on both servers, run the NFS service:

/etc/init.d/nfs restart

Then, mount the NFS drive in your webserver:

mount -t nfs database.com:/usr/local/atmail/users /usr/local/atmail/users nfs

mount -t nfs database.com:/usr/local/atmail/tmp /usr/local/atmail/tmp nfs

You can then run the install.php file of @Mail, which will automatically create the Maildir directory.


Filed under: Multiserver, Linux version — info @ 1:41 am

 

April 4, 2007

 High-traffic @Mail SMTP Setup

For high-traffic installations of @Mail (installation that get more than 5 messages a second; 300 a minute), the SMTP processes can be slow. You may decide to use an external Spam-filtering/Antivirus machine; if so, we recommend that you implement Exim via the following steps:Streamline Exim:

- open up your /usr/local/atmail/mailserver/configure file;

- add the following line at the top:

queue_only

- save changes, then find the following line:

timeout_frozen_after = 7d

- change this to:

timeout_frozen_after = 2d

- save changes, then go to the WebAdmin.

- in WebAdmin > Services > SMTP Settings, set "Max Queue Processes" to "1"

- afterwards, set "Remote Parallel Processes" to "1"

- turn off AV in Filters > Antivirus Settings

- turn off SpamAssassin in Filters > Anti-Spam Settings

- kill all Exim processes, and start it via the following command:

% killall -9 exim
% /usr/local/atmail/mailserver/bin/exim -bd -q15m

Put the Exim db directory in a Ramdisk:

- execute the following in the terminal:

% mount -t tmpfs /dev/shm /usr/local/atmail/mailserver/spool/db/

- verify the permissions:

% chown -R atmail /usr/local/atmail/mailserver/spool/db

This will then create a mount on the db directory via the Virtual Memory mount; ultimately making access times faster.

Miscellany:

- it is generally best to have Exim and MySQL reside on the same server;

- you can further improve the performance by a cronjob-run Exim -q implementation:

--snip--

#!/usr/bin/perl

my $smtp = `ps auxww | grep "exim -q"`;

if(!$smtp) {
print "SMTP queue running...n";
system("/usr/local/atmail/mailserver/bin/exim -q");
}

--snip--

- benchmarks have been run for this implementation on a 64-bit 1.8 Ghz Core 2 machine, 2 GB of RAM, 7200 RPM IDE HD. Peaks have reached 4,000 messages and 20,000 connections a minute. Performance averaged at 50 messages a second.


Filed under: Multiserver, Linux version — info @ 7:10 am

 

June 8, 2006

 Multi server @Mail using DNS-round robin

When configured in a multi-server environment ( http://support.atmail.com/multiserver.html ), @Mail can be installed in a cluster to serve requests for POP3/IMAP, SMTP and Webmail sessions between multiple machines. Using a simple DNS round-robin configuration you can configure the services to load balance between multiple machines.

To proceed you must edit the DNS zone file for your domain, in this example the Bind/named nameserver will be used with the sample domain atmail.org. A similar procedure can be used for other nameserver configurations.

For a sample configuraiton we have 3 machines in the cluster. server1.atmail.org ( 66.45.163.146 ), server2.atmail.org ( 66.45.163.147 ) and server3.atmail.org ( 66.45.163.148 ) - All three machines have @Mail installed as per the installation procedure, and are using a shared mySQL database and NFS storage point for messages.

Firstly the MX records for atmail.org can be setup to point to all three machines. Via the MX records in the DNS server you need to specify a priority for the record, and mail-servers will generally connect in order for the MX priority. For the multi-server cluster we will configure all 3 servers to share the same MX prioirty, allowing inbound mail to be distributed between each server.

First each machine must be defined in the DNS zone file

server1         IN      A       66.45.163.146
server2         IN      A       66.45.163.147
server3         IN      A       66.45.163.148

This points server1, server2 and server3.atmail.com to the prospective IP addresses

Next the DNS MX record can be defined to point to each machine, note 0 is used for the MX priority

atmail.org.           IN      MX      0       server1
atmail.org.           IN      MX      0       server2
atmail.org.           IN      MX      0       server3

This allows inbound messages to be routed to either machine in the cluster.

Next, for POP3/IMAP and Webmail services the domain mail.atmail.org is used for clients to connect via their external mail-client or viewing the Webmail interface. Using a similar approach with the DNS zone file, we can point the mail subdomain to resolve to each IP, allowing the remote client to cycle between each of the machines in the cluster. Note, if a machine is down in the cluster mail will be automatically routed to the next in order.

mail  IN      A       66.45.163.146
mail  IN      A       66.45.163.147
mail  IN      A       66.45.163.148

After applying the changes to the DNS zone file, restart the nameserver and verfiy the DNS changes are working. Using the "dig" utility under Linux you can query the nameserver and check the MX records are resolving correctly:

# dig atmail.org mx

;; QUESTION SECTION:
;atmail.org.                    IN      MX

;; ANSWER SECTION:
atmail.org.             86400   IN      MX      0 server3.atmail.org.
atmail.org.             86400   IN      MX      0 server1.atmail.org.
atmail.org.             86400   IN      MX      0 server2.atmail.org.

Note the DNS server returns the MX records resolves to all three servers. This allows inbound messages to be received by any machine in the cluster.

Next, using the nslookup utility under Linux or Windows, check the mail.atmail.org resolves to each IP successfully

# nslookup mail.atmail.org
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   mail.atmail.org
Address: 66.45.163.146
Name:   mail.atmail.org
Address: 66.45.163.147
Name:   mail.atmail.org
Address: 66.45.163.148

Once verified you can now begin using each of the 3 services in the cluster to accept Inbound/Outbound SMTP, POP3, IMAP and Webmail services. Note, if one of the machines is down in the cluster and the client tries to connect, the protocol will automatically cycle to the next IP address that resolves the domain for a successful connection.


Filed under: Multiserver — Ben Duncan @ 1:28 am

 

December 22, 2004

 Rsync Setup Instructions (Multiserver)

rsync is a powerful tool that allows you to synchronize the contents of your servers (i.e., configuration files, spamassassin .cf filters and the like). They are specially useful if you are doing the multiserver installation of @Mail, and want to load-balance the stress across multiple servers.

This article details the process of creating an Rsync master and a client. For illustrative purposes in this example, the Rsync master hostname will be called "Master-Foo-01" - the Client hostname will be called "Slave-Foo-02":

1.) Create a SSH public key on the Master server:

atmail@master-foo-01# ssh-keygen -t rsa

2.) This will produce a public key in /home/atmail/.ssh/id_rsa.pub. Copy the .pub file to the Slave machine:

atmail@master-foo-01# scp /home/atmail/.ssh/id_rsa.pub root@slavae-foo-02:/home/atmail/.ssh/id_rsa.pub

3.) Pipe the public key into a file called authorized_keys, in your .ssh directory.

atmail@slave-foo-02# cat /home/atmail/.ssh/id_rsa.pub >> authorized_keys

4.) Make sure that the authorized_keys file, and all of the contents of the .ssh directory have 600 permissions:

atmail@slave-foo-02# chmod 600 /home/atmail/.ssh/*

5.) Try to login, sans password, from the Master Machine to the Client machine.

atmail@master-foo-01# ssh atmail@slave-foo-02

6.) If it logins successfully, go on to instruction 7. Otherwise, redo the key-gen process, or check the permissions of your SSH directory.

7.) Create the file "/home/atmail/rsyncservers.sh" in the master server, that contains the following:

#!/bin/sh
/usr/bin/rsync -e ssh -avr --delete /usr/local/atmail/webmail/ atmail@slave-foo-02:/usr/local/atmail/webmail/
/usr/bin/rsync -e ssh -avr --delete /usr/local/atmail/mailserver/configure atmail@slave-foo-02:/usr/local/atmail/mailserver/configure
/usr/bin/rsync -e ssh -avr --delete /usr/local/atmail/mailserver/etc/ atmail@slave-foo-02:/usr/local/atmail/mailserver/etc/
/usr/bin/rsync -e ssh -avr --delete /usr/local/atmail/av/etc/ atmail@slave-foo-02:/usr/local/atmail/av/etc/
/usr/bin/rsync -e ssh -avr --delete /usr/local/atmail/spamassassin/etc/ atmail@slave-foo-02:/usr/local/atmail/spamassassin/etc/

8.) Give it execute permissions:

atmail@master-foo-01# chmod 755 /home/atmail/rsyncservers.sh

9.) Place the following entry in /etc/crontab:

01,10,20,30,40,50 * * * * atmail /home/atmail/rsync-servers.sh

Your rsync service should now check every 10-minutes for altered files, and synchronize when necessary.


Filed under: Multiserver — John Contad @ 8:45 am

 

November 18, 2004

 NFS Mount Checkup Script

After you followed the steps in http://support.atmail.com/multiserver.html#5 to do an NFS
mount of the users directory, NFS dies. Randomly. For no
apparent reason, other than the fact that it wants to die.
Kamikaze NFS. (Actually this is probably due to network-level
problems - harass your local Cisco-certified "engineer")

So what do you do? Make an NFS-checkup script that checks NFS every minute, and resurrects it if found dead. Here's what you'll need to do:

1.) Add an entry in /etc/crontab, that calls a script we will name (aptly) "check-nfs.pl":

01-59 * * * * root /usr/local/atmail/webmail/modules/check-nfs.pl check-nfs.pl

2.) Create the script. It should contain the following:

#!/usr/bin/perl

$mount = `df -k`;

if($mount !~ /[dbserver]:/usr/local/atmail/users/)    {

$date = localtime();
print "NFS DOWN";

system("mount -t nfs [dbserver]:/usr/local/atmail/users /usr/local/atmail/users");

}

3.) Save all changes, and give the appropriate permissions for the file.


Filed under: Multiserver — John Contad @ 3:14 am