Pages

Graphite carbon-cache IOError with too many open files

No comments:
When running carbon-cache daemon, clients are seeing errors such as connection refused by the daemon, a common reason for this is setting small number of file descriptors.

/var/log/carbon/console.log file, there may be

exceptions.IOError: [Errno 24] Too many open files: '/var/lib/graphite/whisper/systems/<host_name>/<metric_name>.wsp'

The number of files carbon-cache daemon can open should be increased. Many Linux systems set file descriptors to a maximum of 1024 as default. A value of 16384 may be good enough depending on how many clients are simultaneously connecting to the carbon-cache daemon.

In Linux, sysctl and ulimit programs can be used to set system-wide resource use.

Listing group membership of a user or members of a group in Linux

No comments:
lid is a handy command-line tool for getting group membership of a user or list of users a group contains.

If you invoke lid just without any option, it will list groups containing the invoking user.

# lid
No user name specified, using root.
root(gid=0)
bin(gid=1)
daemon(gid=2)
sys(gid=3)
adm(gid=4)
disk(gid=6)
wheel(gid=10)

By default lid lists groups containing user name.

# lid mysql
mysql(gid=27)

with -g option lid lists users in a group.

# lid -g users
games(uid=12)

If you don't want to display user or group IDs then you should use -n or --onlynames options.

# lid -g -n bin
bin
root
daemon

Linux whatis command and definitions of some commands in /bin and /sbin directories

No comments:
Linux whatis command searches the whatis database for complete words.

See some command descriptions from whatis database:

addpart (8) - simple wrapper around the add partition ioctl

agetty (8) - alternative Linux getty

arch (1) - print machine hardware name (same as uname -m)

arp (7) - Linux ARP kernel module

arp (8) - manipulate the system ARP cache

arping (8) - send ARP REQUEST to a neighbour host

audispd (8) - an event multiplexor

auditctl (8) - a utility to assist controlling the kernel's audit system

auditd (8) - The Linux Audit daemon

Strict IPTables Rules for postgresql server (Configured to make streaming replication)

No comments:
IPTables rules script for a postgresql server which is configured as a master or a standby for streaming replication.
#!/bin/sh
# IP address of this server
SERVER_IP=$(/sbin/ifconfig -a | awk '/(cast)/ { print $2 }' | cut -d':' -f2 | head -1)

DNS_SERVER=<write IP address of the dns server>
SSH_CLIENT=<write the IP address from where you make ssh connections>
PGE_SERVER=<write IP address of the other postgresql server>

# Flush iptables rules
iptables -F
iptables -X

# Set default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP

# Allow traffic on loopback adapter
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow incoming ssh only
iptables -A INPUT -p tcp -s $SSH_CLIENT -d $SERVER_IP --sport 513:65535 --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT 
iptables -A OUTPUT -p tcp -s $SERVER_IP -d $SSH_CLIENT --sport 22 --dport 513:65535 -m state --state ESTABLISHED -j ACCEPT

# Allow incoming icmp only
iptables -A INPUT -p icmp --icmp-type 8 -s $SSH_CLIENT -d $SERVER_IP -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT 
iptables -A OUTPUT -p icmp --icmp-type 0 -s $SERVER_IP -d $SSH_CLIENT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow incoming postgresql connections
iptables -A INPUT -p tcp -s 0/0 -d $SERVER_IP --sport 1024:65535 --dport 5432 -m state --state NEW,ESTABLISHED -j ACCEPT 
iptables -A OUTPUT -p tcp -s $SERVER_IP -d 0/0 --sport 5432 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT

# Allow outgoing postgresql connections
iptables -A OUTPUT -p tcp -s $SERVER_IP -d $PGE_SERVER --sport 1024:65535 --dport 5432 -m state --state NEW,ESTABLISHED -j ACCEPT 
iptables -A INPUT -p tcp -s $PGE_SERVER -d $SERVER_IP --sport 5432 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT

# Allow outgoing DNS requests
iptables -A OUTPUT -p udp -s $SERVER_IP --sport 1024:65535 -d $DNS_SERVER --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT 
iptables -A INPUT -p udp -s $DNS_SERVER --sport 53 -d $SERVER_IP --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT 
iptables -A OUTPUT -p tcp -s $SERVER_IP --sport 1024:65535 -d $DNS_SERVER --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT 
iptables -A INPUT -p tcp -s $DNS_SERVER --sport 53 -d $SERVER_IP --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT

# drop all other traffic
iptables -A INPUT -j DROP
iptables -A OUTPUT -j DROP

Preserving Linux Shell History Even If Working with Multiple Terminals

No comments:
If you are continuously running shell commands on more than one linux terminal, probably you want all of the shell (mostly bash) prompts to remember any command from any terminal. With the following environmental variables to save the .bashrc file, you can do it so.


# This is for ignoring duplicate entries
export HISTCONTROL=ignoredups:erasedups

# This is for large history
export HISTSIZE=102400

# This is for a big history file
export HISTFILESIZE=100000

# This is for appending commands to history file
shopt -s histappend

# This is for saving and reloading the history after each command is run
export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"

Preserving links in Linux

No comments:
Linux commands like tar and cp have some options that control whether symbolic links are followed or not. When you run tar command which is backing up directories contain multiple links to big files, you would get unnecessary copies of the same data. 

In the case of a cp command if a symbolic link is encountered, the data inside of the file to which the link targets is copied when -L (dereference) option used. But if you use -d (no dereference) option, cp would copy the link itself.

Look at the following example;

Setting Up a Workgroup Directory in Linux

No comments:
The following procedure may be useful to create workgroup folder for a team of people.

The workgroup name is HR and has some members cbing, mgeller, rgreen
The folder is /data/hr
Only the creators of files in /data/hr folder should be able to delete them.
Members shouldn't worry about file ownership, and all members of the group need full access to files.
Non-members should not have access to any of the files.

The followings will match the requirements written above: