“Loomiere 2.0 teaser” reloaded

Filed under: Networking, Projects

«Update… Today this machine reached 6500+ active concurrent connections for a total traffic of 2841.19 Mbps (2778.1 output + 63.09 input) while the SSD’s were still below 20% load. Now that’s what I call serious FUN! Here are some pictures…

…end!»

Since my last post about describing the performance of Loomiere in production, quite a few things have happened. I tested a lot and found many things I would like to improve or add, I tried many hardware configurations to see what goes well with what. In short, I have a whole list of things I want to implement as soon as possible and the most important item on this list is to actually launch Loomiere to the public as soon as possible!

However, here is a quick update on the performance of Loomiere, but this time with a nice twist: it’s running on a SSD-based machine! Here are some specs:

• Intel Core I7 3GHz
• 12 GB RAM
• 6 x 256GB SSD Kingston
• 3 x 1 Gigabit Ethernet cards
• Ubuntu Lucid Lynx with 2.6.37-12-server kernel

And here are the results after only a few hours from launch (graphs follow):

• 4500+ active oncurrent connections (streams)
• 300+ MB/s of throughput (aprox. 2500 Gbps)

However, at this point we hit a wall: the mainboard or the kernel. I have to dig into this problem further, but from what I could gather so far, it seems that the mainboard or the Linux kernel can’t handle more punishment. Following are the CPU and memory graphs; it is clear that while the memory has no worries, the CPU is hogging! The problem lies somewhere between the I/O operations and the actual sending of data to the sockets (interrupts maybe?). More testing will follow regarding this issue.

Hmm… this is a deep one.

So, that’s it for now, I’m going to sleep. Good night .

Loomiere/Stream – A high performance streaming server

Filed under: Networking, Projects

The Loomiere (0.2.1) code is now freely available under GPLv3.
Please see this post for an update.

Are you killing psstream?

Well, yes! I am sure that many of you already know about psstream (the PHP streaming extension I made a while back). Well, many things happened since then and I came to realize I could do better; a whole lot better actually. As of now the ‘psstream’ project is officially no longer developed (see below). It will remain on the website for some time to come for archiving purposes but that is it.

But wait, why did you do it?

For some time now I have be looking into improving the streaming mechanism for a large video-sharing project run by my company. PSStream was a first effort, and it did the job but soon ran into problems. None of our servers was able to properly stream more than 150 clients simultaneously and the resources were grossly wasted, hence, Loomiere/Stream. Read the rest of this entry »