Mail Server Mail Relaxer: Hibernation while running DSM Mail Server and Mail Station

[Basalt]

Hi all,

I developed a standard DSM package called "Mail Relaxer" that does allow your Syno to hibernate

When no e-mail client is active anymore, it will stop some processes of the standard DSM Mail Server and Mail Client packages.
And start them again as soon as an e-mail client becomes active.

Will only work if you use external mail servers (POP3), i.e. not if you run the mail server yourself for your own domain.
Have a look at SourceForge for details.

Best regards,
Erik.

 

[Basalt]

FYI:
I started a similar topic on the English Synology Forum.

/Erik

 

[wweichselg]

Hi Erik,

works your tool with DSM 4.1? Or is DSM 4.2 needed?

Best regards, Wolfgang

 

[Basalt]

I developed it on DSM 4.1, made no significant changes for 4.2, so I expect it to still work on 4.1.
Made a contraint in the installer: DSM >= 4.1

/Erik

 

[KlausHa]

Hello Erik,
thank you very much for your work and this tool!
I did not knew before, that with 7zip you can extract spk files and check the scripts... I had a look on your scripts, but to be honest I have not understood everything.

I have installed the mail_relaxer-1.0.spk on my DS213 (DSM 4.1) via the package-center. I have tested your tool, but unfortunaly my DS does not go into hibernate. :/
Maybe you can give me a hint how to debug? Or you have an idea what I do wrong?

Here is my configuation
hibernate is configured with 10min. Of course I have tested before with switched off mailserver, that the system really is going into hibernate mode.
Now, mailserver and mailstation are running. In the roundcube web-interface the system is configurated in a way, that mails are downloaded with pop3 from my freemail account. The mail-clients I have configurated in a way that they get the mails synhronized via IMAP from my mailserver on the DS213. Also Pop3 is allowed for the clients.
I made sure that no client is connected (means: I closed all mail-clients in my home network: on my PC and tablet.) But DS is not going into hibernate.

After 37min I connected via ssh to the DS and I entered the command "ps" to get a little idea what is going on (see output below)

My interpretation of the output is:
relax_mailprocd is running --> good, so the mail_relaxer is installed and running
syno_mailserverd and dovecot are running --> not so good, so it did not work, that the mail_relaxer has switched off the mailserver processes. That's why the DS can not go into hibernate mode.


PID USER VSZ STAT COMMAND
1 root 2976 S init
2 root 0 SW [kthreadd]
3 root 0 SW [ksoftirqd/0]
4 root 0 SW [events/0]
5 root 0 SW [khelper]
8 root 0 SW [async/mgr]
93 root 0 SW [sync_supers]
95 root 0 SW [bdi-default]
96 root 0 SW [kblockd/0]
102 root 0 SW [ata/0]
103 root 0 SW [ata_aux]
118 root 0 SW [rpciod/0]
126 root 0 SW [kswapd0]
127 root 0 SW [nfsiod]
280 root 0 SW [scsi_eh_0]
282 root 0 SW [scsi_eh_1]
296 root 0 SW [mtdblockd]
325 root 0 SW [kstriped]
336 root 0 SW [md0_raid1]
342 root 0 SW [md1_raid1]
428 root 0 SW [jbd2/md0-8]
429 root 0 SW [ext4-dio-unwrit]
1586 root 0 SW [khubd]
1590 root 0 SW [kethubd]
1649 root 0 SW [ksnapd]
1717 root 0 SW [crypto/0]
1885 root 0 SW [crypto]
1886 root 0 SW [crypto_ret]
1985 root 0 SW [ecryptfs-kthrea]
2518 root 2976 S /sbin/syslogd -S
2523 root 2976 S /sbin/klogd
2617 root 0 SW [md2_raid1]
2856 root 0 SW [kdmflush]
3002 root 8448 S < /usr/syno/bin/findhostd
3179 root 0 SW [jbd2/dm-0-8]
3180 root 0 SW [ext4-dio-unwrit]
3809 root 16912 S scemd
3975 root 0 DW [xhci_thread]
4277 root 0 SW [scsi_eh_2]
4281 root 0 SW [usb-storage]
4509 root 0 SW [usbhid_resumer]
4619 root 2980 S /sbin/getty 115200 console
4639 admin 33756 S /usr/syno/pgsql/bin/postgres -D /var/services/pgsql --config_file=/usr/syno/pgsql/etc/postgresql.conf --hba_file=/usr/syno/pgsql/etc
4641 admin 33756 S postgres: writer process
4642 admin 33756 S postgres: wal writer process
4844 root 8968 S /usr/syno/sbin/hotplugd
4999 root 5436 S /usr/sbin/inetd
5005 root 5436 S /usr/sbin/crond
5013 root 13404 S /usr/syno/sbin/snmpd -Ln -c /usr/syno/etc/snmpd.conf -p /var/run/snmpd.pid udp:161,udp6:161,tcp:161,tcp6:161
5146 root 8272 S /usr/syno/sbin/cupsd -C /usr/local/cups/cupsd.conf
5204 root 12884 S N /usr/syno/sbin/fileindexd
5214 root 27400 S N /usr/syno/sbin/synoindexd
5219 root 26308 S N /usr/syno/bin/synomkthumbd
5309 root 19376 S /usr/syno/sbin/nmbd -D
5343 root 25692 S /usr/syno/sbin/smbd -D
5372 root 25692 S /usr/syno/sbin/smbd -D
5632 root 0 SW< [kslowd000]
5791 root 0 SW< [kslowd001]
6176 root 7468 S /usr/syno/sbin/cnid_metad -l log_error
6181 root 14372 S /usr/syno/sbin/afpd -g guest -c 256 -n DataStation:AFPServer -l default log_error
6437 1 1584 S /sbin/portmap
6440 root 0 SW [lockd]
6441 root 0 SW [nfsd4]
6442 root 0 SW [nfsd]
6443 root 0 SW [nfsd]
6444 root 0 SW [nfsd]
6445 root 0 SW [nfsd]
6468 root 8292 S /usr/sbin/statd
6561 root 8316 S /usr/sbin/mountd -p 892
6628 root 14076 S /usr/syno/bin/rsync --daemon
6659 root 12988 S /usr/syno/bin/synonetbkpd
6751 root 26308 S N /usr/syno/sbin/synomkflvd
6824 root 12472 S /usr/syno/sbin/synosnmpcd
6858 root 11340 S /usr/syno/sbin/sshd
7028 root 13524 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/httpd.conf-sys
7786 root 2408 S avahi-daemon: running [DataStation.local]
7960 root 4792 S /usr/sbin/openvpn --daemon --cd /usr/syno/etc/synovpnclient/openvpn --config client_o1367103605
7966 root 0 Z [openvpn]
7967 root 4792 S /usr/sbin/openvpn --daemon --cd /usr/syno/etc/synovpnclient/openvpn --config client_o1367103605
8399 root 1700 S /var/packages/VPNCenter/target/sbin/pptpd -c /usr/syno/etc/packages/VPNCenter/pptp/pptpd.conf -o /usr/syno/etc/packages/VPNCenter/pp
8410 root 51976 S /var/packages/VPNCenter/target/sbin/vpnauthd
8430 root 2976 S /bin/sh /tmp/relax_mailprocd boot_start
8750 root 8180 S N /usr/syno/bin/photostationd
8862 root 11168 S /var/packages/MailServer/target/sbin/dovecot
8865 root 15336 S dovecot-auth
8866 root 9492 S /var/packages/MailServer/target/sbin/syno_mailserverd
8905 dovecot 11064 S pop3-login
8906 dovecot 11064 S pop3-login
8907 dovecot 11064 S pop3-login
8908 dovecot 11072 S imap-login
8909 dovecot 11072 S imap-login
8910 dovecot 11072 S imap-login
9000 root 97712 S /usr/syno/apache/bin/httpd -DHAVE_PHP
9002 nobody 97712 S /usr/syno/apache/bin/httpd -DHAVE_PHP
9003 nobody 97712 S /usr/syno/apache/bin/httpd -DHAVE_PHP
9004 nobody 98056 S /usr/syno/apache/bin/httpd -DHAVE_PHP
9037 root 13764 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/httpd.conf-sys
9298 root 13780 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/httpd.conf-sys
12708 root 0 SW [flush-9:0]
12709 root 0 SW [flush-253:0]
12821 root 8216 S /usr/sbin/ntpd -p /var/run/ntpd.pid -g
12960 nobody 97712 S /usr/syno/apache/bin/httpd -DHAVE_PHP
12976 root 13772 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/httpd.conf-sys
12977 root 13772 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/httpd.conf-sys
13051 root 13764 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/httpd.conf-sys
13052 root 13764 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/httpd.conf-sys
13258 root 14524 S sshd: admin [priv]
13278 root 14656 S sshd: admin@pts/0
13279 admin 5440 S -sh
13684 root 26336 S /usr/syno/sbin/smbd -D
13774 root 2976 S sleep 3
13776 admin 5440 R ps


Where I can add some traces into the relax_mailprocd, that I have a chance to debug your tool? (e.g. some traces to have a look when start or stop commands are performed by the mail_relaxer and also check the return codes of the same commands...) Maybe you have already a debug version available with the trace-outputs?

By the way, I have also some basic questions - in case I need to add some trace outputs:
After adding the traces, can I simply zip it with a zip tool and rename as "apk"? And than I can simply re-install? Or is there a special other process/tooling involved for creating the .apk?
(sorry for this basic question - you see, I have done some little scripting in past, but I am not an linux or synology expert )
Or: can I simply copy relax_mailprocd on the ds and can I start it from the ssh-console without installing over package center?
what would be the command to start it with which parameters?

Regards,
KlausHa

P.S. one more hint: I usually working on windows xp...

 

[KlausHa]

While I wrote this long posting above, the system went into hibernate... But I really did not expected this. It went into hibernate only after ~1,5h or ~2h
I openend a mail-client and the DS waked up automatically - great!
Now I closed the client again to check if my DS goes into hibernate. But after 10min the DS is still on. I am tired now, I go to bed...

Only one more question, which came to my mind: do you have implemented any delay between switching off the last client and sending the stopp command to fetchmail and postfix?

Argh, I guess I found the problem:
I switched off all things like virus-scanner or backupjobs on my DS. But the DS of my parents did a backup job to my DS as a target via DSL... I did not thought about this... sorry.

However I am tired, I will re-check tomorrow.
Sorry for confusion.

 

[Basalt]

Hello KlausHa,

There is a delay of approx 10 minutes before mail processes will be stopped.
This is done to prevent unnecessary starts/stops.
Starting is much faster, mererly depends on the startup speed of the mail processes themselves.

For me as well, first time after install (or startup?) it took longer, actually I don't know why.

If you want to debug, I suggest you to stop the package with Package Center, change the script(s), and restart the package again.
See image on SourceForge to find where files are stored.

In Package Center -> Mail Relaxer you can find the [View Log] link, that will open /var/log/messages where all logging is done.
Once you are happy with the changes you can update the SPK package (or convince me to do so )

BR,
Erik.

 

[KlausHa]

Hello Eric,
thank you for your hints!
today I made some measurements by adding some traces.

My observations:
1.)There is a process called dovecot-auth -w. It takes ~10min or 11min, that this process gets killed (after no clients are active). I made 2 measurements and it took almost the same time. (But from your code I have not understood why it is 10 or 11min...)
Only after IsActiveMailClient is setted to 0 (because than you say no webclient is active)
Hint: I have not opened the roundcube webinterface.

2.)After your big loop with 200 cycles is starting. Since you have at the end a "sleep 3" command. It takes exactly 10min. And only after related processes are killed

3.) after the "hibernate counter" of the diskstation is starting, which is set in my configuration to the minimum value of 10min.

--> consequence at least 30min you have to wait until the diskstation goes into hibernate. But in reality it took 40min until my diskstation went into hibernate. This is strange, I have no idea from where the additional 10min are coming from...

My change

I have done following change: I set the counter from 200 to 20. So after dovecot-auth -w is killed (which takes 10min), only 1 additional minute you are in this additional loop.

Questions:
-do you see any risk (e.g. unkown strange side-effects when setting the countercheck from 200 to 20)?
-You have any idea, how dovecot-auth -w can be killed earlier (e.g after 3min of inactivity of an client instead of 10min)?
--> my goal would be that after 3 or 4min of inactivity of all clients all relevant processes are killed.

P.S. script is really good documented! Thanks a lot for this . So today I understood much more how your script is working. Good idea!

[edit:21.23, 04.07.2013] I do not know if this is important or not: in the roundcube webinterface I configurated, that every 1minute it should be checked if any new mails available.
KlausHa

[edit 21.45, 04.07.2013] with my little change my diskstation went now into hibernate after 17 or 18min, after all clients are closed. This is ok for me, I think I can live with this solution.

 

[KlausHa]

Hello Eric,

With Thunderbird client on PC (windows xp) everything is working fine. Thunderbird closes all imap processes, when it gets closed.
(I double checked with the command "pidof imap" via ssh on the diskstation)

But when I use the mail-client app "K-9 mail" from my android 4.2 tablet I see that this client opens 3 to 5 imap processes in parallel.
These processes are not killed even after manually stopping the app in android.

Is there any automatic "garbage collection" in the DSM? (I mean will the imap processes killed somehow automatically after some time, if the mail-client will not close/kill them?). After how much time such processes would be killed? ( I was not patient enough simply to wait for some hours...).

Is there any possibiliy to implement this kind of "garbage collection" inside a script.
I thought about something like this:

every 20min:
- foreach pidof imap:
if (pid last usage>10min) kill pid

(pid means process id)
What do you think about this? Is it possible to find out the last usage of an imap process in unix?


Regards,
KlausHa

 

[Basalt]

I thought about something like this:

every 20min:
- foreach pidof imap:
if (pid last usage>10min) kill pid

Don't know if this can be done.
However, I would expect that the imap processes would die at some point, when not used anymore.
Maybe a little bit more patience ?

About your earlier question: yes, you can make it faster. I have chosen for 10 minutes because I tend to close Thunderbird accidently when I only want to close one message window. Then the mail processes should not stopped directly, because I restart Thunderbird right away.

Also, it is better not to hibernate/wakeup to much, since harddisk spin down/up should not be done too often each day. I have a DSM hibernation timeout of 1 hours.

Best regards,
Erik.

/Erik.

 

[MC_Bergsteiger]

Hello Erik,
thanx for your effort; your script is exactly what I am looking for.
But as I am a newcomer, I am not sure if your limitation "NOT suitable when you run your own domain on MailServer" holds for me.

My situation is as follows:
- I have a registered domain "mydomain.com" which provides e-mail services, too (eg. "myname@mydomain.com" is my e-mail address).
- I use Syno Mail Server only for IMAP (to my thunderbird clients) and SMTP relay. In the SMTP configuration dialog I have to enter "mydomain.com" in the "domain name:" field. Otherwise I get errors when I try to send mails.
- Mail Station (Roundcube) is polling my e-mails from the mail server of my domain provider via pop3.
- I am running the Syno DNS server in order to resolve the hostname of my Syno rackstation when I am in my local network. (This way I can connect to "myhostname.mydomain.com" no matter if I am at home or in an external network.)

When all e-mail clients are off, I get the notification "relax: INFO: Mail clients inactive, stopping mail processes" in "/var/log/messages". Mail "pickups" are indeed stopped after this.

What still prevents my Synology from hibernation are frequent (every 5 minutes) activites by the mailserver daemon, e.g:
Jul 17 04:14:33 kernel: [YZ.ZY] [/var/run/dovecot/master.pid] opened by pid XYZ [u/var/packages/MailServer/target), commsyno_mailserver)]
[Update: Very surprisingly, I get these activities even if I uncheck "SMTP" and "IMAP" services. So, even if my MailServer has nothing to do at all, it shows these frequent activities. Any suggestions why this is the case?]

Is this caused by "running my own domain on the mail server"?
So, can't your script prevent these activities?
Why is that?
As said above, I do not use my Syno mail server as a "full mail server" for the domain, but only for IMAP and SMTP relay purposes.

I would be very thankful if you or any other member could explain to me,
1) why I have to enter "mydomain.com" in the domain name field, although I am just using SMTP relay
2) why it is not possible to force the mail server to stop, although it is used for an own domain.

Best regards,


Matthias

 

[Basalt]

Hi Matthias,
I understand that you are not running your own mail server, but poll mails from an external mail server, which serves your personal domain mails. That is indeed how it should be, if you want to hibernate. Actually, this is how I have it as well (additionally, I poll from a hotmail account).

You always have to enter something as domain name, I guess this is because the internal smtp server function is combined with the external mail server function. I have "Basalt" as domain name, being my Syno server name.

I don't know why your mail server still seems active, maybe try stopping one of the related packages, just for testing?

/Erik

 

[MC_Bergsteiger]

Hi Erik,

thank you very much for your explanations. Now I understand that I was on the right way regarding this topic.

Your suggested solution also worked fine. It was as easy as this: stopping and restarting the mail server (without changing any configuration) resolved the issue...

Now, my rackstation is sleeping well again ;-)

Many greetings


Matthias

 

[Basalt]

Good to hear it is working for you too
BR,
Erik.

 

[nachon]

I think the tool is not for me because i am using zarafa, but: Great Job i think.

 

[Basalt]

I think the tool is not for me because i am using zarafa, but: Great Job i think.

I'm not using Zarafa and not planning to do so either.
Nevertheless I am pretty sure the package will need to be changed if used with Zarafa.

Basically one needs to determine which processes are started when an e-mail client is active (either via webmail or imap) and which processes need to be stopped to allow hibernation. Knowing that, one can adapt the package and try to make it work for Zarafa.

Best regards,
Erik

 

[nachon]

When will you Release it over the Package Center in the Diskstation?

 

[Basalt]

Don't know how to do that, and if I should anyway.
Manual installation via Package Center is easy, just:

1. on your PC, downloaded the package from SourceForge
2. in DSM Package Manager, use the [Manual install] button and browse to the location where you have put the download

Best regards,
Erik.

 

[Basalt]

Good to mention:
Credits for the original idea of Mail Relaxer go to Tobi "jahlives" from German Synology forum.
(added this to description on SourceForge as well)

/Erik

 

[MC_Bergsteiger]

Hello Erik, dear all,

do you have any experience with Mail Relaxer or the mail services, respectively, under DSM 4.3-3776-2?

Although Mail Relaxer shuts down the mail services as expected I still have a lot of activity caused by mail services, which prevents my rackstation from hibernation.

This is /var/log/messages just before (!) Mail Relaxer woke up the mail services:

Oct  7 06:26:49 SRV kernel: [395173.968329] [.] opened by pid 21017 [u:(MailScanner: waiting for messag), comm:(MailScanner)]
Oct  7 06:26:50 SRV kernel: [395175.863215] [/etc/ld.so.cache] opened by pid 32364 [u:(pidof), comm:(pidof)]
Oct  7 06:26:53 SRV kernel: [395178.901934] [/etc/ld.so.cache] opened by pid 32368 [u:(pidof), comm:(pidof)]
Oct  7 06:26:55 SRV kernel: [395179.975788] [.] opened by pid 21017 [u:(MailScanner: waiting for messag), comm:(MailScanner)]
Oct  7 06:26:57 SRV kernel: [395181.940636] [/etc/ld.so.cache] opened by pid 32372 [u:(pidof), comm:(pidof)]
Oct  7 06:27:00 SRV kernel: [395184.979352] [/etc/ld.so.cache] opened by pid 32376 [u:(pidof), comm:(pidof)]
Oct  7 06:27:03 SRV kernel: [395188.018069] [/etc/ld.so.cache] opened by pid 32380 [u:(pidof), comm:(pidof)]
Oct  7 06:27:06 SRV kernel: [395191.056797] [/etc/ld.so.cache] opened by pid 32384 [u:(pidof), comm:(pidof)]
Oct  7 06:27:09 SRV kernel: [395194.095507] [/etc/ld.so.cache] opened by pid 32388 [u:(pidof), comm:(pidof)]
Oct  7 06:27:12 SRV kernel: [395197.134222] [/etc/ld.so.cache] opened by pid 32392 [u:(pidof), comm:(pidof)]
Oct  7 06:27:15 SRV kernel: [395200.172937] [/etc/ld.so.cache] opened by pid 32396 [u:(pidof), comm:(pidof)]
Oct  7 06:27:18 SRV kernel: [395203.210660] [/etc/ld.so.cache] opened by pid 32400 [u:(pidof), comm:(pidof)]
Oct  7 06:27:21 SRV kernel: [395206.249378] [/etc/ld.so.cache] opened by pid 32404 [u:(pidof), comm:(pidof)]
Oct  7 06:27:24 SRV kernel: [395209.288080] [/etc/ld.so.cache] opened by pid 32408 [u:(pidof), comm:(pidof)]
Oct  7 06:27:27 SRV kernel: [395212.326784] [/etc/ld.so.cache] opened by pid 32412 [u:(pidof), comm:(pidof)]
Oct  7 06:27:30 SRV kernel: [395215.365508] [/etc/ld.so.cache] opened by pid 32416 [u:(pidof), comm:(pidof)]
Oct  7 06:27:33 SRV kernel: [395218.403246] [/etc/ld.so.cache] opened by pid 32420 [u:(pidof), comm:(pidof)]
Oct  7 06:27:36 SRV relax: INFO: Mail clients active, restarting mail processes

This behaviour can be seen all night, although nobody accesses the rackstation.

When I check, what processes "pidof" is looking for, I get the following:

/bin/pidof fetchmail
/bin/pidof imap
/bin/pidof dovecot-auth

Is this a bug in DSM 4.3-3776-2?
Or can you adjust Mail Relaxer to prevent also this behaviour?

Stopping and restarting all packages and the whole rackstation did not resolve the issue.
Synology support was not able to help me. They said I should re-install the whole rackstation without any 3rd party apps first...

Any ideas?

Cheers,


Matthias

P.S.: I am currently running a RS812+ with DSM 4.3-3776-2 and the following packages
- Audiostation
- Cloudstation
- DDNS-Updater
- DHCP-Server
- DNS-Server
- Init 3rd Party
- Mail Relaxer
- Mail Server
- Mail Station
- Medienserver
- Perl
- Photo Station
- Video Station
- IPKG