A friend of mine recently asked me how to limit bandwidth per process in Linux. Here are the steps for accomplishing this using iptables and tc. The idea is to create a traffic class, specify the class priority and the bit-rate etc, tell the traffic class that packets with a handle [x] should be handled by this class, and assign packets a handle using iptables. iptables allows to assign a handle using gid or pid. Here are the commands. These commands do not limit incoming traffic so, one has to set iptables for INPUT accordingly.
#setting the root queuing class (can be ignored)
tc qdisc add dev eth0 root handle 1: htb default 15
#specify the class with priorty 1. The class id is 1:1
tc class add dev eth0 parent 1: classid 1:1 htb rate 10kbit ceil 10kbit prio 1
#specify the filter. so packets with handle '10' go into the class 1:1
tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 10 fw classid 1:1
#now give the packets a handle of 10
iptables -t mangle -A OUTPUT -m owner --gid-owner [gid] -j MARK --set-mark 0xa
A good description of of traffic shaping in Linux can be found here.