mysqldump script (per tables)


#!/bin/bash

# This is PER TABLE backup. Each table will be backed up individually

## to enable passwordless mysqdump, put your password in /etc/mysql/[mysqld|percona|maria].conf.d/client.conf
# [client]
# user=""
# pass=""

THEDB="mydbname" # THE DATABASE
OUTPUT="/home/backup/mysqldump"
DATE=`date +%Y%m%d`
OPTS="--max_allowed_packet=512M "

tables=`mysql -e "use $THEDB; show tables;" | tr -d "| " | grep -v -E "^Tables_in_" `
mkdir -p $OUTPUT/$THEDB-$DATE

for table in $tables; do
STAGEDDATE=`date +%Y%m%d`
echo "Dumping table : " $table
mysqldump -a $OPTS $THEDB $table > $OUTPUT/$THEDB-$DATE/$STAGEDDATE-$table.sql
gzip $OUTPUT/$THEDB-$DATE/$STAGEDDATE-$table.sql
done

echo "Backup done at " $OUTPUT/$THEDB-$DATE

Web server tuning (apache and nginx)

Key points

  • enable http2 & change mpm prefork to event
  • php-fpm

 

Enable http2

On apache + prefork to event – https://techwombat.com/enable-http2-apache-ubuntu-16-04/

sudo add-apt-repository ppa:ondrej/apache2
sudo apt update
sudo apt upgrade
sudo apt install php7.0-fpm 
sudo a2enmod proxy_fcgi setenvif
sudo a2enconf php7.0-fpm 
sudo a2dismod php7.0 
sudo a2dismod mpm_prefork 
sudo a2enmod mpm_event 
sudo service apache2 restart
sudo service php7.0-fpm restart

Add in <VirtualHost>… </VirtualHost> for individual site, or in apache.conf file, for global settings.

Protocols h2 h2c http/1.1

sudo a2enmod http2
sudo service apache2 restart

Nginx – Need to compile nginx with http2 module
./configure –with-compat –add-dynamic-module=../ModSecurity-nginx –with-http_ssl_module –with-stream_ssl_module –prefix=/etc/nginx –with-http_v2_module

** mod_security for nginx, follow this : https://www.nginx.com/blog/compiling-and-installing-modsecurity-for-open-source-nginx/

 

This post content is under development.. new content will be added in the future.

Elasticsearch notes

Useful links

Elasticdump – http://blog.ruanbekker.com/blog/2017/11/22/using-elasticdump-to-backup-elasticsearch-indexes-to-json/

sudo npm install n -g
sudo n stable

 

https://www.digitalocean.com/community/tutorials/how-to-interact-with-data-in-elasticsearch-using-crud-operations

Delete by time
https://hobo.house/2016/02/18/how-to-manually-clean-indexes-from-elasticsearch/

 

Slow HTTP Dos Verify and mitigate

What it is
https://en.wikipedia.org/wiki/Slowloris_(computer_security)

Tools to check
Slowloris : https://github.com/gkbrk/slowloris
SlowHttpTest : https://github.com/shekyan/slowhttptest

Mitigatation

mod_qos
apt-get update && apt-get -y install libapache2-mod-qos && a2enmod qos && /etc/init.d/apache2 restart
* configuration file in /etc/apache2/mods-enabled/

mod_reqtimeout
a2enmod reqtimeout && /etc/init.d/apache2 restart
* configuration file in /etc/apache2/mods-enabled/

Securing apache

Some notes on securing apache..

A few key points

  • Disable access to .
  • Disable banner
  • disabled PHP functions
  • Slow dos mitigation (link)

Disable access to .
https://stackoverflow.com/questions/4352737/apache-configuration-regex-to-disable-access-to-files-directories-beginning-wit

<filesmatch "^\.ht">
Order allow,deny
Deny from all

<filesmatch “^\.”>
Order allow,deny
Deny from all

<directorymatch “^\.|\/\.”>
Order allow,deny
Deny from all

Disable banner
http://www.ducea.com/2006/06/15/apache-tips-tricks-hide-apache-software-version/

Usually found in /etc/apache/conf-enabled/security.conf

Disable PHP functions
https://www.cyberciti.biz/faq/linux-unix-apache-lighttpd-phpini-disable-functions/

disable_functions =exec,passthru,shell_exec,system,proc_open,popen,pcntl_exec,curl_exec,curl_multi_exec,parse_ini_file,show_source,phpinfo,eval,assert,create_function

Slow DOS mitigation
http://www.farhanfaisal.com/slow-http-dos-verify-and-mitigate/

Easiest way to install ffmpeg on mac os

Just keeping reference.

https://www.oodlestechnologies.com/blogs/Easiest-Way-To-Install-FFmpeg-On-Mac-OS-X
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

https://gist.github.com/clayton/6196167
brew install ffmpeg --with-vpx --with-vorbis --with-libvorbis --with-vpx --with-vorbis --with-theora --with-libogg --with-libvorbis --with-gpl --with-version3 --with-nonfree --with-postproc --with-libaacplus --with-libass --with-libcelt --with-libfaac --with-libfdk-aac --with-libfreetype --with-libmp3lame --with-libopencore-amrnb --with-libopencore-amrwb --with-libopenjpeg --with-openssl --with-libopus --with-libschroedinger --with-libspeex --with-libtheora --with-libvo-aacenc --with-libvorbis --with-libvpx --with-libx264 --with-libxvid

Installing cuckoo sandbox on Mac OS

Cuckoo sandbox is an automated malware analysis system. Its does utilize virtualization engine to isolate malware execution and analysis. You will be given a web-interface, as well as CLI tools to communicate with cuckoo, such to upload, and reviewing all reports.

I’ve found a complete tutorial on this.

http://advancedmalwareprotection.blogspot.com/2012/03/installing-cuckoo-on-max-os-x-lion.html

Hope this would help anyone to have this. The tutorial is a bit technical. If you’re not familiar with Mac system or Terminal, you might want just to use any public sandbox.

Smokeping: Ping latency grapher to the rescue

Smokeping is a great tool, written by Tobias Oetiker, which is the same guy who wrote MRTG, the world most popular graphing tool. Smokeping can gives you another dimension in viewing your network environment, that is ping latency, and graph nicely with nice colours.

Installation were easy, you can install it using apt-get in Debian-based distro. The configurations were done using the config file, which is usually located in /etc/smokeping/. The default configuration files includes some samples, which allow you to follow the syntax to add additional hosts to monitor.

The great thing about Smokeping, it tells you link quality metric, which usually being done manually with ping. The ping data will be graph as the sample graph here, which could tell you the ping latency (time it took for ping until response), and also dropped packet.

smokeping

As the example above, we can see that during the day, the link quality dropped to almost 95% during the day, and latency increases as well. Even though you have a high speed link, this might tell a different story, which might caused by dropped packets at network devices during transit, probably it could not handle the huge traffic.