AlienVault OSSIM SSH Access

Today I was troubleshooting an issue and found something absolutely maddening on the default build of AlienVault’s OSSIM server. The SSH server doesn’t start up until VERY late in the boot process.

lrwxrwxrwx 1 root root 27 Jun 22 2017 S01alienvault-depmod -> ../init.d/alienvault-depmod
lrwxrwxrwx 1 root root 17 Jun 22 2017 S01mongodb -> ../init.d/mongodb
lrwxrwxrwx 1 root root 14 Jun 22 2017 S01motd -> ../init.d/motd
lrwxrwxrwx 1 root root 17 Jun 22 2017 S01netdiag -> ../init.d/netdiag
lrwxrwxrwx 1 root root 23 Jun 22 2017 S01open-vm-tools -> ../init.d/open-vm-tools
lrwxrwxrwx 1 root root 15 Jun 22 2017 S01ossec -> ../init.d/ossec
lrwxrwxrwx 1 root root 25 Jun 22 2017 S01rabbitmq-server -> ../init.d/rabbitmq-server
lrwxrwxrwx 1 root root 17 Jun 22 2017 S01rsyslog -> ../init.d/rsyslog
lrwxrwxrwx 1 root root 17 Mar 1 07:58 S02apache2 -> ../init.d/apache2
lrwxrwxrwx 1 root root 17 Mar 1 07:58 S03openvpn -> ../init.d/openvpn
lrwxrwxrwx 1 root root 15 Jun 22 2017 S04acpid -> ../init.d/acpid
lrwxrwxrwx 1 root root 14 Jun 22 2017 S04cron -> ../init.d/cron
lrwxrwxrwx 1 root root 16 Mar 1 07:58 S04fprobe -> ../init.d/fprobe
lrwxrwxrwx 1 root root 20 Jun 22 2017 S04irqbalance -> ../init.d/irqbalance
lrwxrwxrwx 1 root root 14 Jun 22 2017 S04logd -> ../init.d/logd
lrwxrwxrwx 1 root root 19 Jun 22 2017 S04memcached -> ../init.d/memcached
lrwxrwxrwx 1 root root 15 Mar 1 07:58 S04mysql -> ../init.d/mysql
lrwxrwxrwx 1 root root 17 Mar 1 07:58 S04nagios3 -> ../init.d/nagios3
lrwxrwxrwx 1 root root 15 Mar 1 07:58 S04nfsen -> ../init.d/nfsen
lrwxrwxrwx 1 root root 22 Jun 22 2017 S04redis-server -> ../init.d/redis-server
lrwxrwxrwx 1 root root 15 Jun 22 2017 S04rsync -> ../init.d/rsync
lrwxrwxrwx 1 root root 23 Jun 22 2017 S04smartmontools -> ../init.d/smartmontools
lrwxrwxrwx 1 root root 16 Jun 22 2017 S04squid3 -> ../init.d/squid3
lrwxrwxrwx 1 root root 13 Mar 4 16:21 S04ssh -> ../init.d/ssh
lrwxrwxrwx 1 root root 17 Jun 22 2017 S04sysstat -> ../init.d/sysstat

While I understand that you should *almost* never need to get SSH access to your AlienVault or OSSIM server, starting SSH after literally everything else is less than ideal. For instance, if there’s a MySQL database corruption issue (that’s what I was experiencing), you have to wait for MySQL to time out and all the services that try to start that depend on MySQL fail out before you can access the machine. Thinking about console access? Surprise – the virtual terminals have been disabled as well.

So, what we did for an immediate fix is to boot from external media and run the following commands:

mkdir /mnt/temp
mount -t ext4 /dev/sda1 /mnt/temp
cd /mnt/temp/etc/rc2.d
ln -s ../init.d/ssh S01alz_ssh
rm S04ssh
cd /
umount /mnt/temp
reboot

The intent is that SSH starts immediately after the depmod process kicks off but before anything else. The rationale here is simple. If anything else has a failure, we shouldn’t need to wait to get in to troubleshoot on the command line.  We then rebooted and had near immediate SSH access rather than having to wait for anything else to start (and in the case of a troubleshooting event, likely fail).