Server has a weak, ephermal Diffie-Hellman public key (Zimbra 7)

Chrome error:

weak dh key error in chrome

Firefox error:

weak dh key error in firefox

The recommended workaround is to update Zimbra, but there’s an easy workaround: Disabling the insecure ciphers.

/opt/zimbra/bin/zmprov mcf +zimbraSSLExcludeCipherSuites TLS_DHE_RSA_WITH_AES_128_CBC_SHA
/opt/zimbra/bin/zmprov mcf +zimbraSSLExcludeCipherSuites TLS_DHE_RSA_WITH_AES_256_CBC_SHA
/opt/zimbra/bin/zmprov mcf +zimbraSSLExcludeCipherSuites SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
/opt/zimbra/bin/zmmailboxdctl restart

Source:
https://wiki.zimbra.com/wiki/Disabling_the_use_of_weak_DH_keys_in_Zimbra_Collaboration_mailboxd

Server has a weak, ephermal Diffie-Hellman public key (Zimbra 7)

ERROR 2006 (HY000) at line 749: MySQL server has gone away (MariaDB/MySQL)

OS:CentOS Linux release 7.1.1503
DB: MariaDB 5.5.44

Importing a big mysql dump file resulted in the following error:

ERROR 2006 (HY000) at line 749: MySQL server has gone away

Line 749 was a very big INSERT query. To fix this, place the following into /etc/my.cnf.d/large.cnf

[mysqld]
max_allowed_packet = 64M
wait_timeout = 6000

[mysqldump]
max_allowed_packet = 64M

Then restart mariadb/mysqld:

systemctl restart mariadb
ERROR 2006 (HY000) at line 749: MySQL server has gone away (MariaDB/MySQL)

Monitoring free inodes on Linux with Nagios/Adagios

This howto assumes:

  • nrpe is installed and working on the client
  • CentOS 6/7 on both sides
  • Nagios/Adagios server with pynag installed and working

On the server you want to monitor:

Install the check_disk plugin for nrpe:

yum install nagios-plugins-disk

Add the following to /etc/nrpe.d/check_disk_inodes.cfg:

command[check_disk_inodes]=/usr/lib64/nagios/plugins/check_disk -W "$ARG1$" -C "$ARG2$" "$ARG3$"

Restart NRPE (NOTE: Use systemctl if using CentOS 7):

service nrpe restart

On the Nagios server:

Add a check command:

pynag add command command_name="2ks-check_nrpe_disk_inodes" command_line='$USER1$/check_nrpe -H $HOSTADDRESS$ -c check_disk_inodes -a "$_SERVICE_WARNING$" "$_SERVICE_CRITICAL$" "$_SERVICE_OPTIONAL_ARGUMENTS$"'

NOTE: In my case pynag placed the cfg file in /etc/nagios/commands/, but it was not included as a cfg_dir in nagios.cfg. To fix that, run:

pynag config --append cfg_dir=/etc/nagios/commands/

Add the service to the host:

pynag add service service_description="Disk inodes" use="generic-service" host_name="host.domain.com" check_command="2ks-check_nrpe_disk_inodes" __CRITICAL="5%" __WARNING="10%"

Reload nagios (NOTE: Use systemctl if using CentOS 7):

service nagios reload

The check output should now show something like:

DISK OK - free space: / 1613 MB (35% inode=95%): /boot 53 MB (57% inode=99%): /dev/shm 1004 MB (100% inode=99%): /var/spool 8682 MB (53% inode=11%):
Monitoring free inodes on Linux with Nagios/Adagios

Atlassian Jira: Dashboard Diagnostics: Mismatched URL Scheme

Problem:

We’ve detected a potential problem with JIRA’s Dashboard configuration that your administrator can correct. Click here to learn more.

When “Click here to learn more” is clicked, it shows:

jira dashboard diagnostics mismatched url scheme

Dashboard Diagnostics: Mismatched URL Scheme
JIRA is reporting that it is using the URL scheme ‘http’, which does not match the scheme used to run these diagnostics, ‘https’. This is known to cause JIRA to construct URLs using an incorrect hostname, which will result in errors in the dashboard, among other issues….

Solution:

Open /opt/atlassian/jira/conf/server.xml with an editor

Find <Connector port=”8081″…./> and add:

scheme="https"
proxyName="jira.domain.com"
proxyPort="443"

It should look similar to this:

jira connector

Restart jira:

service jira stop
service jira start

Jira should stop complaining about the URL Scheme now.

If you get an error about mismatching port scheme, you forgot to add proxyPort=”443″.

Source: https://confluence.atlassian.com/display/JIRA061/Integrating+JIRA+with+Apache+using+SSL#IntegratingJIRAwithApacheusingSSL-Step1:ConfigureTomcat

Atlassian Jira: Dashboard Diagnostics: Mismatched URL Scheme

Creating custom okconfig templates

For this example I have a host (google.com) with HTTP, HTTPS, DNS, and Ping checks.

google-1

I’ve customized some of the service checks and want to create a template called “Google Server” from this host and it’s services.

To do this I will have to combine the config files for all services into a template. Templates are by default stored in /etc/nagios/okconfig/examples/, and have the file extension .cfg-example.

Locate all config files for this host, this can be done in a few ways, but the easiest is probably with pynag:

[root@adagios ~]# pynag list --quiet filename where host_name=google.com and register=1 | sort | uniq
/etc/nagios/okconfig/hosts/default/google.com-dns.cfg
/etc/nagios/okconfig/hosts/default/google.com-host.cfg
/etc/nagios/okconfig/hosts/default/google.com-http.cfg
/etc/nagios/okconfig/hosts/default/google.com-https.cfg
[root@adagios ~]#

I can do this in the Adagios web interface too, by searching for the host, selecting all services and clicking the bulk edit button (I won’t actually be editing anything, bulk edit will just show me all file names).
google-2

Next I want to combine all services into a template file: /etc/nagios/okconfig/examples/google-server.cfg-example

[root@adagios ~]# cat /etc/nagios/okconfig/hosts/default/google.com-dns.cfg > /etc/nagios/okconfig/examples/google-server.cfg-example
[root@adagios ~]# cat /etc/nagios/okconfig/hosts/default/google.com-http.cfg >> /etc/nagios/okconfig/examples/google-server.cfg-example
[root@adagios ~]# cat /etc/nagios/okconfig/hosts/default/google.com-https.cfg >> /etc/nagios/okconfig/examples/google-server.cfg-example

NOTE: I didn’t include the host config because I haven’t defined any custom services in it. If there are any services in the host config you would like in the template (chances are there will be), add them to the template file yourself. To see what services are defined in the host config, use: pynag list where object_type=service and register=1 and filename=<location of host config>

[root@adagios ~]# pynag list where object_type=service and register=1 and filename=/etc/nagios/okconfig/hosts/default/google.com-host.cfg
object_type          shortname            filename
--------------------------------------------------------------------------------
service              google.com/Ping      /etc/nagios/okconfig/hosts/default/google.com-host.cfg
service              google.com/test      /etc/nagios/okconfig/hosts/default/google.com-host.cfg
----------2 objects matches search condition------------------------------------
[root@adagios ~]#

As seen above, I added a “test” service to the host through the Adagios web interface, and it was saved in the host config file. The define service {…} part is what you would add to the template config.

[root@adagios ~]# cat /etc/nagios/okconfig/hosts/default/google.com-host.cfg
...
define service {
         service_description            test
         use                            generic-service
         host_name                      google.com
        check_command                 okc-check_dummy
        __EXIT_CODE                   0
        __MESSAGE                     Cool!
}

To prepare the template so it can be applied to other hosts, replace the host name and group name with HOSTNAME and GROUP. Okconfig will substitute HOSTNAME and GROUP with the host (and if any, group) name you specify when adding the template. In my case the hostname was google.com and group was default.

sed -i 's/google.com/HOSTNAME/g;s/default/GROUP/g' /etc/nagios/okconfig/examples/google-server.cfg-example

Example how a service defenition should change:

Before:

define service {
        host_name               google.com
        contact_groups          default
        service_description     HTTPS google.com
        check_command           okc-check_https
        use                     okc-check_https
        __URI                   /
        __SEARCH_STRING
        __RESPONSE_WARNING      2
        __RESPONSE_CRITICAL     10
        __VIRTUAL_HOST          google.com
        __PORT                  443
}

After:

define service {
        host_name               HOSTNAME
        contact_groups          GROUP
        service_description     HTTPS HOSTNAME
        check_command           okc-check_https
        use                     okc-check_https
        __URI                   /
        __SEARCH_STRING
        __RESPONSE_WARNING      2
        __RESPONSE_CRITICAL     10
        __VIRTUAL_HOST          HOSTNAME
        __PORT                  443

}

Next, add the host and select the newly created Google Server template:

google-3

google-4

I deleted the google.com host, removed all config files, and added it again with only the template I created:

google-5

All this could probably be done in one (or a few) pynag copy commands, but I haven’t tested that.

Creating custom okconfig templates

iodine client/server on CentOS 7

From http://code.kryo.se/iodine/:

iodine lets you tunnel IPv4 data through a DNS server. This can be usable in different situations where internet access is firewalled, but DNS queries are allowed.

DNS Setup

Name     Type    Value            
iodine   NS      tunnel.domain.com
tunnel   A       123.123.123.123

Server:

Install:

yum install iodine-server -y

Configure iodine-server.service in “/etc/sysconfig/iodine-server”:

OPTIONS="-f -P 'good password' 172.21.21.1/24 iodine.domain.com"

where 172.21.21.1/24 is the tunnel ip/netmask

Start the server:

[root@iodine ~]# systemctl start iodine-server.service
[root@iodine ~]# systemctl status iodine-server.service
iodine-server.service - Iodine Server
 Loaded: loaded (/usr/lib/systemd/system/iodine-server.service; enabled)
 Active: active (running) since Sat 2015-06-20 02:24:28 GMT; 42s ago
 Main PID: 1960 (iodined)
 CGroup: /system.slice/iodine-server.service
 └─1960 /usr/sbin/iodined -f -P 172.21.21.1 24 iodine.domain.com

Jun 20 02:24:28 iodine.domain.com systemd[1]: Starting Iodine Server...
Jun 20 02:24:28 iodine.domain.com systemd[1]: Started Iodine Server.
Jun 20 02:24:28 iodine.domain.com iodined[1960]: Opened dns0
Jun 20 02:24:28 iodine.domain.com iodined[1960]: Setting IP of dns0 to 172.21.21.1
Jun 20 02:24:28 iodine.domain.com iodined[1960]: Setting MTU of dns0 to 1130
Jun 20 02:24:28 iodine.domain.com iodined[1960]: Opened IPv4 UDP socket
Jun 20 02:24:28 iodine.domain.com iodined[1960]: Listening to dns for domain iodine.domain.com
Jun 20 02:24:28 iodine.domain.com iodined[1960]: started, listening on port 53

Enable IPv4 forwarding in the kernel:

echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.d/99-sysctl.conf
sysctl -p /etc/sysctl.d/99-sysctl.conf

Client:

Install:

yum install iodine-client

Configure iodine-client.service in “/etc/sysconfig/iodine-client”:

iodine -f -r 123.123.123.123 iodine.domain.com -P 'good password'

Start the client:

[root@iodine-client ~]# systemctl start iodine-client
[root@iodine-client ~]# systemctl status iodine-client
iodine-client.service - Iodine Client
 Loaded: loaded (/usr/lib/systemd/system/iodine-client.service; disabled)
 Active: active (running) since Sat 2015-06-20 02:27:46 GMT; 3s ago
 Main PID: 2020 (iodine)
 CGroup: /system.slice/iodine-client.service
 └─2020 /usr/sbin/iodine -f -r 123.123.123.123 iodine.domain.com -P

Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Using EDNS0 extension
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Switching upstream to codec Base128
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Server switched upstream to codec Base128
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: No alternative downstream codec available, using default (Raw)
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Switching to lazy mode for low-latency
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Server switched to lazy mode
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Autoprobing max downstream fragment size... (skip with -m fragsize)
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: 768 ok.. 1152 ok.. 1344 ok.. 1440 ok.. 1488 ok.. 1512 ok.. 1524 ok.. will use 1524-2=1522
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Setting downstream fragment size to max 1522...
Jun 20 02:27:46 iodine-client.domain.com iodine[2020]: Connection setup complete, transmitting data.

Test client -> server and server -> client ping:

[root@iodine-client ~]# ping 172.21.21.1
PING 172.21.21.1 (172.21.21.1) 56(84) bytes of data.
64 bytes from 172.21.21.1: icmp_seq=1 ttl=64 time=0.045 ms
64 bytes from 172.21.21.1: icmp_seq=2 ttl=64 time=0.054 ms
64 bytes from 172.21.21.1: icmp_seq=3 ttl=64 time=0.038 ms
64 bytes from 172.21.21.1: icmp_seq=4 ttl=64 time=0.057 ms
^C
--- 172.21.21.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.038/0.048/0.057/0.010 ms
[root@iodine-client ~]#
[root@iodine ~]# ping 172.21.21.2
PING 172.21.21.2 (172.21.21.2) 56(84) bytes of data.
64 bytes from 172.21.21.2: icmp_seq=1 ttl=64 time=0.063 ms
64 bytes from 172.21.21.2: icmp_seq=2 ttl=64 time=0.062 ms
64 bytes from 172.21.21.2: icmp_seq=3 ttl=64 time=0.064 ms
64 bytes from 172.21.21.2: icmp_seq=4 ttl=64 time=0.057 ms
^C
--- 172.21.21.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.057/0.061/0.064/0.008 ms
[root@iodine ~]#

Example of an ssh tunnel (socks proxy) through the iodine server:

ssh 172.21.21.1 -p 443 -D 8080 -f -N

Or if you are using the iodine NetworkManager plugin:

dnf install NetworkManager-iodine-gnome -y

Screenshot from 2015-06-20 01-58-57

Screenshot from 2015-06-20 01-59-14

Screenshot from 2015-06-20 02-00-44

Screenshot from 2015-06-20 02-02-43

Screenshot from 2015-06-20 02-02-48

 

iodine GitHub: https://github.com/yarrick/iodine

 

iodine client/server on CentOS 7

Hardened OpenVPN on CentOS 7

This post should cover installing and hardening OpenVPN, configuring firewalld to allow VPN traffic, and configure logrotate to rotate the OpenVPN logs on CentOS 7.

Consider reading this first: https://community.openvpn.net/openvpn/wiki/Hardening

SELinux should be enforcing and key permissions should not allow anyone but root to read them.

Installing

First install the EPEL repo:

yum install epel-release -y

Update the system:

yum update -y

Install openvpn and easy-rsa:

yum install openvpn easy-rsa -y

Copy the easy-rsa scripts to /etc/openvpn/easy-rsa:

cp -R /usr/share/easy-rsa/2.0/ /etc/openvpn/easy-rsa/

Copy the OpenVPN sample server config to /etc/openvpn:

cp /usr/share/doc/openvpn-2.*/sample/sample-config-files/server.conf /etc/openvpn/

Edit the following variables in the /etc/openvpn/easy-rsa/vars file:

export KEY_COUNTRY="US"
export KEY_PROVINCE="CA"
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="mail@domain"
export KEY_EMAIL=mail@domain

Edit the KEY_SIZE variable to increase the key size to something above 3072 (4096 is probably not a bad idea unless you suffer performance problems):
https://community.openvpn.net/openvpn/wiki/Hardening#X.509keysize
https://www.enisa.europa.eu/activities/identity-and-trust/library/deliverables/algorithms-key-sizes-and-parameters-report

KEY_SIZE=4096

Create the server side keys and certificates:

cd /etc/openvpn/easy-rsa/
source vars
./clean-all
./build-ca
./build-key-server server

Build the Diffie-Hellman parameters:
Note: this will take a long time, in some cases more than an hour. Consider installing and starting haveged before doing this.

./build-dh

OpenVPN won’t start if the CRL file doesn’t exist or is invalid, so we create a dummy client certificate and revoke it:

./build-key dummy-client
./revoke-full dummy-client

When dropping the OpenVPN daemon privileges after initialization to “nobody”, it won’t be able to read the crl.pem file because /etc/openvpn/easy-rsa/keys has 0700 permissions. We work around this by moving it to /etc/openvpn/crl.pem and symlinking to /etc/openvpn/easy-rsa/keys/crl.pem. This way we don’t have to make /etc/openvpn/easy-rsa/keys world-readable or edit the revoke-full script. Nice.

mv /etc/openvpn/easy-rsa/keys/crl.pem /etc/openvpn/crl.pem
ln -s /etc/openvpn/crl.pem /etc/openvpn/easy-rsa/keys/crl.pem

Generate a client certificate/key combo:

cd /etc/openvpn/easy-rsa/
source vars
./build-key client1

Generate a TLS pre-shared key:

cd /etc/openvpn/easy-rsa/keys
openvpn --genkey --secret ta.key

Edit the server configuration file /etc/openvpn/server.conf:

vim server.conf

Certificate Authority, Server Certificate, Server Key:

ca easy-rsa/keys/ca.crt
cert easy-rsa/keys/server.crt
key easy-rsa/keys/server.key # This file should be kept secret

Diffie-Hellman parameters:

dh easy-rsa/keys/dh4096.pem

Push a default gateway route:

push "redirect-gateway def1 bypass-dhcp"

Push DNS options:

push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 208.67.220.220"

Use a TLS authentication secret:
https://community.openvpn.net/openvpn/wiki/Hardening#Useof–tls-auth

tls-auth easy-rsa/keys/ta.key 0 # This file is secret

Maximum number of concurrently connected clients (change this if you have more than 10 clients):

max-clients 10

Drop privileges after initialization:

user nobody
group nobody

Append log:

log-append /var/log/openvpn.log

Check the Extended Key Usage on the certificates:

Note: The –remote-cert-tls client option is equivalent to –remote-cert-eku “TLS Web Client Authentication”

remote-cert-tls client

Check for revoked certificates:

crl-verify crl.pem

Set a minimum TLS protocol version:

tls-version-min 1.2

Set a stronger cipher:

cipher AES-256-CBC

Use SHA-2 for message authentication:
Note: The source blog says “SHA-256”, but OpenVPN wouldn’t start unless I changed it to SHA256.

I changed this to SHA512 because why not. Use SHA256 if you suffer performance problems. See Algorithms, Key Sizes and Parameters Report – 2013 (3.3.1 Recommended Hash Functions, page 26).

auth SHA512

Limit the list of supported TLS ciphersuites:
https://community.openvpn.net/openvpn/wiki/Hardening#Useof–tls-cipher

tls-cipher TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256:TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256:TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384:TLS-DHE-RSA-WITH-AES-256-CBC-SHA256

The final server config

And a corresponding client config (see server config explanations above for the same directives).

Replace the dots (“…”) in the inline tags with the corresponding certs/keys:

<ca> = certificate authority (contents of: /etc/openvpn/easy-rsa/keys/ca.crt)
<cert> = client certificate (contents of: /etc/openvpn/easy-rsa/keys/client1.crt)
<key> = client private key (contents of: /etc/openvpn/easy-rsa/keys/client1.key)
<tls-auth> = pre-shared tsl key (contents of: /etc/openvpn/easy-rsa/keys/ta.key)

In the client configuration, verify the server certificate subject string.
For example:

verify-x509-name 'C=XX, ST=NA, L=XX, O=XX, OU=XX, CN=XX, name=XX, emailAddress=XX' subject

To see these values for the server certificate, use:

Note: The string must match the subject, but the text output from openssl puts forward slashes between the CN, name, and emailAddress fields. These should be separated by “, ” as shown above. Otherwise you will get an error stating that the subject doesn’t match.

To generate the subject string:

openssl x509 -in easy-rsa/keys/server.crt -text|grep Subject:|sed 's|/name=|, name=|g;s|/emailAddress=|, emailAddress=|g;s|.*Subject: ||g'
Subject: C=US, ST=CA, L=SanFrancisco, O=Fort-Funston, OU=MyOrganizationalUnit, CN=server, name=EasyRSA, emailAddress=me@myhost.mydomain

Enable and start the OpenVPN service:

systemctl enable openvpn@server
systemctl start openvpn@server

Note: the @server means systemd will start openvpn with the config file “server.conf”.
For multiple servers/clients use systemctl enable openvpn@server2, systemctl enable openvpn@client1, etc..

Firewall and forwarding

Enable IPv4 forwarding in the kernel:

/etc/sysctl.d/99-sysctl.conf:

net.ipv4.ip_forward=1
sysctl -p /etc/sysctl.d/99-sysctl.conf

iptables:
Note: Replace ens18 with your internet-facing interface

iptables -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp -m state --state NEW -m udp --dport 1194 -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -s 10.8.0.0/24 -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o ens18 -j MASQUERADE

firewalld/firewall-cmd:

**todo**

Logrotate:

Put the following in /etc/logrotate.d/openvpn:

/var/log/openvpn.log {
 missingok
 notifempty
 copytruncate
 compress
 delaycompress
 daily
 rotate 7
 create 0600 root root
}

useful commands:

View effective config without comments or other garbage:

egrep -iv "^(\#|;|$)" server.conf | sort

Sources and further reading:

https://blog.g3rt.nl/openvpn-security-tips.html
https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage
https://community.openvpn.net/openvpn/wiki/Hardening
https://wiki.mozilla.org/Security/Server_Side_TLS#Modern_compatibility
https://www.enisa.europa.eu/activities/identity-and-trust/library/deliverables/algorithms-key-sizes-and-parameters-report
http://darizotas.blogspot.com/2014/04/openvpn-hardening-cheat-sheet.html

Hardened OpenVPN on CentOS 7