Managing SIP extensions with puppet

I decided to take the next step into the world of Puppet and actually created my own Provider and Type. So I would like to introduce "puppet-asterisk" which creates a custom type and provider for managing Asterisk's SIP configuration. The provider allows for simple SIP resource management, by managing /etc/asterisk/sip.conf and treating each SIP extension defined as an individual resource, so the following extension:

[100]
type=friend
host=dynamic
secret=MyPass123
context=internal
mailbox=100@default
callgroup=1
pickupgroup=1
dtmfmode=rfc2833
canreinvite=no
permit=10.34.0.1/32
deny=10.34.0.2/32

Can be expressed as:

sip {'100':
    ensure  => present,
    type    => 'friend',
    host    => 'dynamic',
    secret  => 'MyPass123',
    context => 'internal'
    mailbox => '100@default'
    permit  => '10.34.0.1/32',
    deny    => '10.34.0.2/32',
}

This allow to actually manage a SIP extension and it's properties individually and even grouping them as Virtual Resources. I have scouted Puppet Forge and most solutions focus completely on uploading the entire sip.conf as a template, which is pretty limiting. We must aim to keep as much as the infrastructure managed by Puppet as possible. Installing At the time of this writing I have not yet submitted it to puppet forge, so you must clone the repository from Github https://github.com/AlexRRR/puppet-asterisk and copy it to your /etc/puppet/modules Testing Please make sure you have rake, rspec installed

gem install rake
gem install rspec

Run the tests from root directory of project

>
rake spec

Creating a private signed repository with Ubuntu

To have a streamlined deployment and management strategy via Puppet it is necessary to be able to create signed Debian Packages and have a private signed repository that can contain such packages, this makes the Puppet manifest simpler and the updates will be done in a controlled manner.

Webserver configuration

Installing the webserver

apt-get install apache

<Directory /var/www/repos/ >
        Options Indexes FollowSymLinks Multiviews
        Order allow,deny
        Allow from all
</Directory>
<Directory "/var/www/repos/apt/*/db/">
        Order allow,deny
        Deny from all
</Directory>
<Directory "/var/www/repos/apt/*/conf/">
        Order allow,deny
        Deny from all
</Directory>
<Directory "/var/www/repos/apt/*/incoming/">
        Order allow,deny
        Deny from all
<?/Directory>

RNG Tools

The rngd daemon acts as a bridge between a Hardware TRNG (true random number generator) such as the ones in some Intel/AMD/VIA chipsets, and the kernel's PRNG (pseudo-random number generator). Virtual Machines are generally really bad at generating enough entropy that the gpg key generation need. RNG Tools aids that process.< /br>

NOTE: This step is optional but highly recommended if you are using a virtual machine

apt-get install rng-tools
vi /etc/default/rng-tools

Edit and Add the following

HRNGDEVICE=/dev/urandom

Finally start the service

/etc/init.d/rng-tools start

GPG key file generation

Next we will generate a GPG key to sign our repository and all of our packages with.

root@agent1:/home/alex# gpg --gen-key
gpg (GnuPG) 1.4.11; Copyright (C) 2010 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 
Key does not expire at all
Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) "
Real name: DevOps stuff
Email address: dev@gmx.de
Comment: DevOps stuff
You selected this USER-ID:
    "devops (DevOps stuff) "
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.
gpg: gpg-agent is not available in this session
passphrase not correctly repeated; try again.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
...+++++
+++++
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
.................+++++
.......+++++
gpg: key F15542E1 marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
pub   2048R/F15542E1 2012-11-30
      Key fingerprint = FF1E A313 F3DA 256B A40E  FB3E 0749 613D F155 42E1
uid                  devops (DevOps stuff) 
sub   2048R/024EC2A2 2012-11-30

Create public GPG key

Now the key is already created and in the keychain. You may list it with

root@agent1:/home/alex# gpg --list-keys
/root/.gnupg/pubring.gpg
------------------------
pub   2048R/F15542E1 2012-11-30
uid                  devops (DevOps stuff) 
sub   2048R/024EC2A2 2012-11-30

Create the gpg key file that will be shared publicly via the webserver.

root@agent1:/home/alex# gpg --armor --output secure.gpg.key --export 024EC2A2

please note 024EC2A2 comes from the gpg --list-keys above!

Configure the Repository

Create the directory structure

mkdir -p /var/www/repos/apt/debian/conf
cd /var/www/repos/apt/debian/conf

Create the file "distributions" in the cwd and make sure the gpg key is used to sign the repository, add the following to the file

Origin: Alex Debian repo
Label: alex
Codename: precise
Architectures: amd64 i386 source
Components: main
Description: alex apt repository
DebOverride: override.precise
DscOverride: override.precise
SignWith: 024EC2A2

Create a file called "options" with the following content:

verbose
basedir .
ask-passphrase

Now create an empty file called "override.precise"

touch override.precise

Signing existing packages

dpkg-sig --sign builder /home/alex/demo-package-2.0.6_1.0-1_amd64.deb

Managing the repository

For this task we will use the tool "reprepro" which makes it very straightforward:
Install reprepro:

sudo apt-get install reprepro

Add a package:

reprepro -Vb . includedeb precise /home/alex/demo-package-2.0.6_1.0-1_amd64.deb

Remove a package:

reprepro -b . remove precise demo-package

Please note that when removing we are using the package name as it is described in the package manifest and the the filename itself

Encryped private keys

Protecting your private ssh-keys

I've noticed that most people are content just generating a keypair via ssh-keygen and using it as it is, but it is extremely important that the private key itself is encrypted. (I am not talking about the passhphrase... I am taking about encryption here). This can be achieved this way

openssl genrsa -aes256 -out raal.pem 2048

This will generate an encrypted .pem file, protected with the password you entered. This pemfile contains both your public and private keys. To extract your public key you need to do this:

chmod 600 raal.pem

ssh-keygen -y -f raal.pem > raal.pub

Counting occurences of subclasses in Django

Counting occurences of subclasses in Django

As a Django learning project I have began CONTMAN, a simple mobile content distribution system, and as expected I have ran into many different kinds of problems that I have thought it would be interesting to keep a log of... and this is it.

This might be the classic case of "When all you have is a hammer everything looks like a nail" as when I was thinking on how to design the project I came across Django inheritance and polymorphism features. "Aha! this is EXACTLY what I need" I said to myself. Now I am not so sure, but I am perhaps one of the most stubborn people in the world, so I kept going in that direction.

So I thought. I have two, maybe three different kinds of content I want to be able to distribute with this platform, for now I have "Ringtones" and "Wallpapers" (NB: I know it is not bleeding edge stuff, but keep in mind this was for learning, and I could not think of anything else at the time), and both of them are Content.

So we have a parent Content Class

class Content(models.Model):
    name = models.CharField(max_length=100)
    created_on = models.DateTimeField('created', auto_now_add = True )
    keyword = models.CharField(max_length=100)

And to children subclasses

class Ringtone(Content):
    artist = models.ManyToManyField(Artist)
    data_file = models.FileField(upload_to="uploads")

class Wallpaper(Content):
    category = models.ForeignKey(Category)
    data_file = models.ImageField(upload_to="uploads")

Ok, so there is a whole bunch of other stuff there but this is enough to get the point of this post across.

To me this was going to be very helpful in a lot of ways to mention some:

• Delivering content would be as easy as pointing to Content.data_file (which you can't is it is clearly stated at the very end of this doc entry.)
• The admin app would let me easily create a manager for uploading the file depending on the type (see comment above)
• Creating reports would be very easy, just create a query set from the Content model.

This last point is the one we will be discussing. 

So it turns out that

a = Content.objects.get(pk=1)

returns a Content object, so you cannot count or fetch content using  "a.data_file".  So how can I do a simple report showing the count of each type of content available? The easy (lame) way could be to simple do two queries of Ringtone and Wallpapers as we only have those and leave Content alone. But this is not nice, in fact it reeks of bad code and self-pity. What if I later add support for "applications" and "games" Content types?

So after a nice session of Django docs + stack overflow + google I think I came across a very simple but effective solution here in stack overflow

The idea is to take advantage of the Contenttypes framework to keep track of what kind of model each object represents. This way we can aggregate easily on our query a create this type of report. So we modify the base (parent) class like this:

class Content(models.Model):
    name = models.CharField(max_length=100)
    created_on = models.DateTimeField('created', auto_now_add = True )
    keyword = models.CharField(max_length=100)
    content_type = models.ForeignKey("contenttypes.ContentType")

With this then we can easily do this:

class Content(models.Model):
>>>Content.objects.values('content_type__name').annotate(hits=Count("content_type"))
[{'hits': 1, 'content_type__name': u'ringtone'}, {'hits': 2, 'content_type__name': u'wallpaper'}]

Mysql and group_concat!

I'm determined to write a little more often on my blog (I've skipped 2008 altogether) because it also works as my personal notepad for the world to see. So let's get started.

Today I've discovered a wonderful new MySQL trick: group_concat and it saved me from doing tedious/slow scripting and also from trying to understand how Stored Procedures work in MySQL (which is something like arameic to me right now).

I've been working with Eventum which is a lovely open source bug tracker from the great thinkers at MySQL AB (now.. Sun). Eventum lets you add N custom fields to the issue creation and listing page, and it adds all of them to a table called eventum_issue_custom_fields. very much like this:


Each custom field has an Id, but they are all entered in different "rows" like this:
(lets say custon field "1" is the tv show name and field "2" is a short description)

IssueId	CustomFieldId	CustomFieldData
50	1	90210
50	2	Lame TV show
51	1	Family Guy
51	2	Funny Stuff

what if i wanted to select from this table the tv show name and the description in just one row?

Thinking about loops in your scripts?
Thinking about stored procedures?
Thinking it cannot be done at all?

Check this out:

SELECT IssueId,GROUP_CONCAT(CustomFieldData ORDER BY CustomFieldData SEPARATOR ',') AS value_list FROM tv_show_table GROUP BY IssueId ORDER BY IssueId

And you will get some nice stuff like:

IssueId	value_list
50	90210, Lame TV Show
51	Family Guy, Funny Stuff

How cool is that?!

the actual eventum query would be:

SELECT icf_iss_id,group_concat(icf_value order by icf_value separator ',') as values_list from eventum_issue_custom_field where icf_fld_id=2 OR icf_fld_id=1 group by icf_iss_id order by icf_iss_id;

Because i only want fields with id 1 or 2.


Bye!

Freetds, DBD-Sybase, Fedora 10

This is an update to an earlier entry

If you want to install DBD-Sybase Perl Module and use freetds in Fedora 10
you must install a patch for it to work. My versions are freetds-0.82, DBD-Sybase-1.09 and of course Fedora 10.

Find your way to the FreeTDS source directory, such that this command

works:



$ grep '\$Id' include/cspublic.h

static const char rcsid_cspublic_h[] = "$Id: cspublic.h,v 1.61 2008/09/08

17:50:25 jklowden Exp $";



Your ID will be different; it should match what's in the patch: 1.58.



Then "man patch" is your friend.  Like it says,



	usually just



	patch 

is all you need, as in:



	$ patch < /path/to/your/cspublic.BLK_VERSION_150.patch



Then build and install per usual.



(shamelessly stolen from http://lists.ibiblio.org/pipermail/freetds/2009q1/024260.html)





Enjoy!!

Solving asterisk´s HDLC Abort Issue

Have you encountered the following msgs in your /var/log/asterisk/messages?

Oct 23 16:54:51 NOTICE[3345] chan_zap.c: PRI got event: HDLC Abort (6) on Primary D-channel of span 1
Oct 23 16:54:54 NOTICE[3345] chan_zap.c: PRI got event: HDLC Bad FCS (8) on Primary D-channel of span 1

if so... you've come to the right place. This is a very tricky one. There are mailing lists with tons of messages regarding this issue scattered throught the web, but none of the them actually follow throught what causes this and how to solve it.

The tricky part is that you system may probably seem to work, you can receive and place calls, but every now and then... all of the calls get dropped at the same time. "Must have been a quirk" you say to yourself even though you know such thing does not exist. Then a few minutes afterward... again. And that YELLOW alarm is now staring you in the eyes with that smug in his face....

PRI configuration is not for the faint of heart, that is for sure. But this message 99 times out of hundred is a harware incompatibility issue, not ISDN config.

High-Level Data Link Control or HDLC is a bit-oriented synchronous data link layer protocol, HDLC was introduced into Digium systems in order to allow data and voice to be used/routed in the same card. This is a good thing, the bad thing is that it relies on more on the system clock for timing. And new motherboards and kernels do not play nice with this settings and old telephony cards.

This may be caused due to IRQ interrupt sharing, APIC, HD params, but to put it plain and simple: YOUR TELEPHONY CARD DOES NOT LIKE YOUR MOTHERBOARD. I wish i had seen that sentence a few months ago.

This was exactly the case with my Proliant ML150 G3 server, which has an Intel Motherboard and specially an Intel Onboard ethernet card. I had a TE110p, which has been my workhorse in several other settings, so it was strange for me that it failed on me so often. As it turns out, Digium's TE110P is the picky girl type, it does not play well with others because of its chipset (TigerJet). Digium was aware of this, so they had to go back to the drawing board and came up with what they like to call "VoiceBus Technology" which basically works a whole lot better.

So here it is, plain and simple if you have a TE110P card having this problems, you MUST update your card to a newer version or get a motherboard that works with that TE110P and hold on to it, hard. They are getting harder to find by the minute. So the solution: Buy a TE120P with the new VoiceBus technology or just run to the Sangoma Camp.


I wish i had a magical bash script that solved this issue... but this time you must take the expensive road.

Luckily you may have found this blog very early in the guessing process of solving this issue and saved a ton of time. Trust me, i was seconds away of trying voodoo or any other sort of black magic.


Hope this helps!


Alex