Post Tagged with: "linux"

Automating Exchange Rate Data

It is often useful to be able to retrieve and update exchange rates in an automated manner. Fortunately, the European Central Bank publishes its foreign exchange reference rates on a daily basis, and they're kind enough to provide an XML file with the data. Although the data uses the Euro as the base currency, we can easily recalculate the rates with US Dollars as the base currency.

For example, to get the current exchange rate for Euros to US Dollars, we can run the following:

curl --silent http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml | grep -e 'USD' | cut -d \' -f 4

As of May 22, 2009, the reference rate returned for US Dollars was 1.3972. To turn the USD per Euro rate of $1.3972/€ into Euro per USD (USD as the base rate), we simply take the inverse. To do so, we calculate 1/$1.3972, which equals €0.7157/$. To perform the calculation programatically and return the Euro per USD value:

echo "scale=4; 1/`curl --silent http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml | grep -e 'USD' | cut -d \' -f 4`" |bc -l

When calculating foreign exchange rates, one must consider the principle of arbitrage, which is expressed mathematically as A-to-C = (A-to-B)*(B-to-C). Therefore, GBP per USD = (GBP per EU)*(EU per USD). To perform the calculation programatically and return the GBP per USD value:

```
#!/bin/bash
```

USD_EU=`curl --silent http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml | grep -e 'USD' | cut -d \' -f 4`

GBP_EU=`curl --silent http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml | grep -e 'GBP' | cut -d \' -f 4`

EU_USD=`echo "scale=5; 1/$USD_EU" | bc -l`

GBP_USD=`echo "scale=5; $GBP_EU*$EU_USD" | bc -l`

```
echo $GBP_USD
```

For example, using the reference rates provided by the European Central Bank as of May 22, 2009, running our script to calculate the GBP per USD rate yields £0.62846/$.

May 24, 2009 0 comments Read More

Debian Lenny apt-cacher reporting bug

With the apt-cacher package in Debian Lenny, my web interface for the cache statistics wasn't being properly updated. Eventually, I tracked down the problem and figured out that I needed to modify the following section of apt-cacher-report.pl to look like this:

```
#parse logfile:
```
```
foreach $logfile_line (@logdata)
```
```
{
```
```
        #$logfile_line =~ s/ /\+/g;
```

        @line = split /\|/, $logfile_line;

```
        $req_date = $line[0];
```
```
#       $req_pid = $line[1];
```
```
#       $req_ip   = $line[2];
```
```
        $req_result = $line[3];
```
```
        $req_bytes  = 0;
```

        $req_bytes  = $line[4] if $line[4] =~ /^[0-9]+$/;

```
#       $req_object = $line[5];
```

May 18, 2009 0 comments Read More

Quick tip: Using sort on large files

When using the sort command on a 2GB file, with a small /tmp (mine was 16M at the time), it failed with the following error…

sort: write failed: /tmp/sort9tRGAQ: No space left on device

I briefly considered that I may need to expand my /tmp partition, until I consulted Google and found a much easier solution – make sort use an alternate location for its temporary files with the -T switch, as follows:

sort -T /path/to/alternate/tmp

May 18, 2009 0 comments Read More

Delete all messages using mutt

Here's a simple, but extremely useful tip if you've ever wanted to delete all your email messages using mutt. In the main view, press shift+d, then enter the following as the pattern to match: ~s .*

May 16, 2009 0 comments Read More

Saving time and bandwidth with apt-cacher

Saving time and bandwidth are two things every system administrator loves. Implementing apt-cacher could save you a considerable amount of both, depending on the number of servers you have. As the name implies, apt-cacher caches packages and package lists for apt packaging systems on Debian or Debian-like systems. Here's how I recently implemented apt-cacher:

apt-get install apt-cacher

I edited /etc/apt-cacher/apt-cacher.conf

cache_dir=/path/to/cache/directory
admin_email=your@email.com
daemon_addr=IPADDRESS

I also edited /etc/default/apt-cacher and set:

AUTOSTART=1

Then ran:

/etc/init.d/apt-cacher start

I imported the existing files apt-get had locally archived:

/usr/share/apt-cacher/apt-cacher-import.pl -r /var/cache/apt/archives

Finally, I edited /etc/apt/sources.list on the apt-cacher server, and each client server:

deb http://hostname:3142/ftp.us.debian.org/debian/ lenny main non-free contrib
deb-src http://hostname:3142/ftp.us.debian.org/debian/ lenny main non-free contrib

deb http://hostname:3142/security.debian.org/ lenny/updates main non-free contrib
deb-src http://hostname:3142/security.debian.org/ lenny/updates main non-free contrib

March 24, 2009 2 comments Read More

Debian NAS using AoE (Part 2)

As I mentioned in Debian NAS using AoE (Part 1), I recently needed to implement a low-cost, enterprise grade storage system for backups and to provide network attached storage for vserver-based Debian servers. I was able to build a suitable storage server with 13TB of raw storage for a reasonable price, and decided to use Debian with LVM2 and AoE to divide and share it across the network.

Before getting to the OS installation, I had to setup and build the RAID arrays in the HighPoint 3540 BIOS. I set up a RAID 1 mirror of the two 300GB drives to use for the OS. Then, I created a 6-disk RAID6 and a 7-disk RAID6, which left me with 4GB and 5GB of usage storage.

Next, I installed Debian Lenny from the Lenny netinst ISO onto the 300GB RAID 1 mirror I created, with no additional packages/package groups selected. After the installation was complete, I rebooted the server to install openssh server, lvm2, and AoE. Since I wanted to be able to use the storage server to run vserver instances if needed, I also installed the vserver kernel and tools:

apt-get update
apt-get install vserver-debiantools linux-image-2.6-vserver-amd64 lvm2 vblade vblade-persist aoe-tools

I configured LVM to recognize the physical volumes:

pvcreate /dev/sdb
pvcreate /dev/sdc

I created the volume groups data1 and data2 on the physical volumes:

vgcreate data1 /dev/sdb
vgcreate data2 /dev/sdc

I created two 1TB LVM logical partitions (/dev/data1/backups, /dev/data1/backups2):

lvcreate -L 1TB -n backups data1
lvcreate -L 1TB -n backups2 data1

I made the ext3 filesystem on them so they would be usable:

mkfs.ext3 /dev/data1/backups
mkfs.ext3 /dev/data1/backups2

I configured AoE (ATA over Ethernet):

modprobe aoe

vblade-persist setup 0 1 eth0 /dev/data1/backups
vblade-persist setup 0 2 eth0 /dev/data1/backups2

vblade-persist start 0 1
vblade-persist start 0 2

Finally, I verified they were up and available using:

aoe-discover
aoe-stat

Output from aoe-stat… :

e0.1 1099.511GB eth1 up
e0.2 1099.511GB eth1 up

To mount an AoE export on a remote server, I used the following (on the remote server):

modprobe aoe
apt-get install aoetools
aoe-discover
aoe-stat (should show the available AoE exports)
mkdir /mountpoint
mount /dev/etherd/e0.1 /mountpoint (replace e0.1 with the appropriate device from aoe-stat)

March 3, 2009 1 comment Read More