Top 5 things to do with cheap Intel Atom servers and Free Software

Intel introduced the low-power, affordable Atom chipset a couple years ago. The earlier models included the Atom 230 and 330, running single and dual-cores, respectively, at 1.6 GHz. Frugal with power at 8 watts, it could even run 64-bit. The first-gen Atom motherboards/chipset kits had two chief weaknesses: they maxed out at 2 GB of RAM, and they used RealTec network-interface cards (at least the Supermicro mboards did). RealTecs sometimes crash Linux when the incorrect driver loads, and the RT NICs don't do jumbo frames. The crashes are patchable but it adds a couple hours to your install. (Do not, repeat not, buy the old models no matter how cheap they are. Stick to the D510 series.)

Intel has since released the Atom D510. The processor is up to a 13-watt draw, runs at about the same speed, and still runs 64-bit. What makes these chipsets better is that they max out at 4 GB of RAM and use genuine Intel Gig NICs that are widely supported and can run jumbo frames. This is a big leap, because the configured price of a fully-built mini-server around the platform remains the same. Supermicro has some great Atom motherboards that can support a home or small office doing various things. (The D525 has since come out. Compare the two here.)

Newegg has some good deals on the Supermicro Atom Rack Server and the tower server for about $70 less. Or you can just buy a motherboard and roll your own. My tower server with D510 runs just under 40 watts with 4 GB of RAM, CD, and hard disk.

What can you do with it? There are a number of Linux and BSD appliances that are free.

1) Run your own telephone switch. FreePBX is an awesome, OS-included version of Asterisk that has a clean web GUI. PBX-In-a-flash is based on FreePBX but has a few extras (nonstandard enhancements like dial weather by airport code and dial your local tide schedule) thrown in. Why would you want to run your own PBX? You can do stupid phone tricks, like really cheap phone service at $.01/minute and $1.50/month per number with Vitelity, for one. Or your own conference line. Or your own trunks over VPNs to other offices. Or just blacklist phone numbers and send telemarketers to their own voice-mail hell.

Keep in mind that to light up your home phone lines, you may need an additional analog card from Sangoma or Digium. (FXS and FXO are different, but can run on the same card.) Once you figure out how cool SIP is, you'll want to replace your house phones with SIP phones. However, you can get started with SIP free using the Xlite soft phone. The iPhone/iPod/iPad version is $7.99. It's perfectly true that you can make phone calls cheap with a Linksys PAP2T or Vonage, but if you're a telephone control freak and want to wage war on telemarketers, Asterisk is for you.

2) Run a great firewall like pfSense. It's based on FreeBSD but has a great Web GUI so you can avoid learning the subtle differences between BSD and Linux. pfSense comes in embedded and full. For the Atom server, running the full version is no sweat. And I run SNORT inline on two interfaces with no issues. A full SNORT IPS signature subscription for home use is just $30/year. And pfSense does not just IPSEC point-to-point VPNs, but also WebVPN. Try that on your Cisco ASA 5505, and you'll be paying a couple grand for the equipment, licenses, and subscription. Check out the plugins for pfSense and you'll see how much you can do with one small box. pfSense 2.0 is almost out of beta and there's a 64-bit version. (Plugins include FreeSwitch, an open-source competitor to (open-source) Asterisk, so if you want, you can even run a PBX on your firewall.)

3) Storage: Run FreeNAS and you'll be able to scale reasonably out of the box. The Atom Motherboard has six SATA slots and two GB NICs that do jumbo frames. Hello iSCSI. With 2TB disks, that's 12 TB of storage. FreeNAS offers CIFS (Windows), Apple (AFP), iSCSI, and NFS mount points. Add a serious SATA RAID card to get even more disk IO performance out of your little Atom box. Obviously, I would recommend the tower server case for FreeNAS because it has room for more disks.

4) Run Apache/PHP/MySQL applications. Run Fedora, CentOS, whatever. You can run your own apps at home on top of your favorite flavor of Linux/BSD. One of my favorites is Gallery2. There's even semi-free apps (free for no enterprise features) like Movable Type (you're reading this on an MT platform), and even run more than one on the same mini-Server.

5) Collect data. Run a weather station. Run a log server. Run a crazy blog and see who hits it with which keywords. Collect IPS events from multiple firewalls and correlate them.

LDIFDE export from list of sAMAccountNames, using vbscript

If you need to export a list of accounts from Active Directory into ldif-format files that will preserve attributes, you can try this. It takes a text list of sAMAccountNames (one per line) and writes out an ldif file for each one. You can easily import the same way by changing the arguments on the exec line and removing the export parameters. You'll also need to fix the line breaks.

'v1.1
' The script will take a text file with usernames (sAMAaccountNames and export them via ldifde to individual files
' named as sAMAccountname.ldf.

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objTextFile = objFSO.OpenTextFile("samaccounts.txt",1)
'On Error Resume Next
Do Until objTextFile.AtEndOfStream
strName = objTextFile.Readline
WScript.Echo "sAMAccountName: " & strName
Set objShell = CreateObject("WScript.Shell")
'you can add/remove attributes from the line below, but be sure to get the quotes right.
Set objScriptExec = objShell.Exec("ldifde -f c:\scripts\export\" & strName & ".ldf -s myDomainController -d ""ou=myOU,ou=Clients,dc=domain,dc=com"" -r ""(sAMAccountName=" & strName & ")"" -l objectclass,dn,c,department,description,displayName,employeeID,extensionAttribute10,extensionAttribute8,extensionAttribute9,givenName,homeDirectory,initials,manager,otherTelephone,physicalDeliveryOfficeName,extension,sn,streetAddress,telephoneNumber,extensionAttribute14,extensionAttribute11,extensionAttribute12,wWWHomePage,sAMAccountName,userPrincipalName,mail,mailnickname,telephoneNumber " )
strResults = objScriptExec.StdOut.ReadAll
WScript.Echo strResults

Loop
set objFile=Nothing

'End

They tried to get my server to join a BotNet

I first noticed a mysterious connection on a netstat:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 1 mrtg.sampas.net:42321 223-118-92-213.server:49153 SYN_SENT

I also noticed Apache had opened a shell:

F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
1 S apache 18005 1 0 76 0 - 1282 - Apr10 ? 00:00:00 sh -i

Soon after, perl became a runaway process, consuming 100% of my CPU time. And I thought /sbin/nologin meant user Apache couldn't just get a shell. I updated zlib from 1.2.2 to 1.2.3 to fix a security hole. up2date -u reports everything is up-to-date. (It did that for my old zlib, too.) I don't see any new holes in my applications, MT and Gallery. I did a Nessus scan with recent updates, and all it showed no holes and one warning. I ran clamscan and it didn't find anything, either. Rkhunter found nothing, and nikto gave me the following output:

+ Server: Apache/2.0.52 (Red Hat)
+ Allowed HTTP Methods: GET,HEAD,POST,OPTIONS,TRACE
+ Apache/2.0.52 appears to be outdated (current is at least Apache/2.2.3). Apache 1.3.33 is still maintained and considered secure.
+ / - TRACE option appears to allow XSS or credential theft. See http://www.cgisecurity.com/whitehat-mirror/WhitePaper_screen.pdf for details (TRACE)
+ /usage/ - Webalizer may be installed. Versions lower than 2.10-09 vulnerable to Cross Site Scripting (XSS). CA-2000-02. (GET)

So now I'm stuck looking through my apache access logs, because that's the only thing exposed to the outside world.

I did a capture just while open my firewall for a couple of minutes, and I saw it try to log in to an IRC channel. Ouch. I've been pwned. Fortunately, my firewall stops my server from being used for attacks, and I was able to block the port range used by the IRC bot.

Finally, Red Hat released a lot of new patches for PHP, and I set PerlTaintCheck On in /etc/httpd/conf.d/perl.conf, which was the real problem: user Apache had started listening on port 80 using Perl, so I couldn't even restart httpd.

Next time, I need to check the logs closer and post a network capture of the login process.

More on How the NSA Tracking System Might Work

The Washington Post's David Ignatius postulated about how the National Security Agency's system might work. In doing so, he provided an excellent example of data mining. What the NSA is trying to do is simple and complex at the same time. The data structure is simple, but the sheer volume makes it complex.

The problem may seem hopelessly complex, but if you use common sense, you can see how the NSA has tried to solve it. Suppose you lost your own cellphone and bought a new one, and people really needed to find out that new number. If they could search all calling records, they would soon find a number with the same pattern of traffic as your old one -- calls to your spouse, your kids, your office, your golf buddies. They wouldn't have to listen to the calls themselves to know it was your phone. Simple pattern analysis would be adequate -- so long as they had access to all the records.

The trouble is, simple pattern analysis isn't that simple when you start trying to code it. You would have a giant data cube, and you would have millions of slices to compare with each other. On the other hand, if you have one target number and have a query that pulls all its callees, you could craft another query that searches for those same numbers. You could then score new numbers based on old queries: each query would have a rating of between 0 and 1 with 1 being just like the original number.

If you have voice matching that can confirm a 1, you could design an artifcial neural network that learns as it targets new numbers. Voice matching would require eavesdropping -- but if you got a score of 1, it would be worth the trouble. This way, your neural network could learn what the score is between 0 and 1 that should trigger voice matching.

Reflexive vs. Recursive Queries and Self-Joins

People (mostly database people) keep asking what I meant by reflexive query in my previous posts. Some thought I was confusing a self-join with a recursive query that would allow my DNS server to answer a DNS query for a domain for which it is not authoritative. What I mean is a query that returns a caller and a callee; and then another query that returns the callee's callees. In a self-join, I can match employers to their managers, since both are in the same table. While the NSA-phone tracking system might use some self-joins, what makes the network part work is getting queries from queries, and jumping from callers to callees. Of course I'd like all the results timestamped, too. Recursive queries are a part of this, and SQL 2005 can do it. If I had a few gigabytes worth of phone data, I'd love to let SQL 2005 loose and see what connections I could see. When I say reflexive, though, I mean that I'm going to use my queries to start other queries, and not just as sub-queries. The recursive part could lead to infinite loops. I wonder if the NSA hit any infinite loops when testing their system. Fortunately, you can specify limits on recursion in SQL 2005.

The scary part of this is what would happen if I had the resources to make it run really fast and tuned it to be as efficient as possible. I could select a target and find all of its connections in a few seconds or a few minutes. The difference between seconds and minutes would make a huge difference. In a system where it takes a couple of seconds to generate results, nobody would notice if I ran a few "unofficial" queries on my friends. If it took a few minutes and precious computer time, then people would notice. Utilitarian ethics.

This begs the question: how much computer time (and tax dollars) does our government use tracking down everyone who calls reporters? Then again, leaking classified information is unethical, but people do it anway. Contextual ethics. Of course, there's also the issue of selective enforcement, but that's a legal issue, not an ethical one.

Thus we're left with utilitarian ethics vs. contextual ethics. Who knew that efficient queries and more processing power could give one type of ethics an advantage over the other?

More Ruby on Rails Security

I do Windows, Unix (Solaris), and Linux (mostly Red Hat). Everyone who's into "open-source" keeps telling me how much more secure it is. I'm a CISSP and I've been installing open-source OSes since I had to know the chipset, IRQ and DMA of the NICs in my box to get networking to work. (The DEC Tulip was my favorite.) When I started working with Solaris 7 and Red Hat 4.x, telnet was enabled by default. I still wonder if telnet was enabled on a Trusted Solaris 7 default install. People who tell me any form of Unix is inherently more secure than any Windows don't seem to be familiar with the Morris worm, the Leshka Sendmail exploit, or BIND vulnerabilities. In fact, just mentioning BIND and sendmail in the same sentence is likely to send your security coordinator into the bunker for the rest of the day. Mind you, I've also seen IIS flaws. Can't we all just get along and implement security best practices on whatever platforms we're using?

Ruby on Rails shows a lot of promise as to helping people get up and running on applications quickly. The tutorials are pretty helpful , but there are a a couple of caveats:

In the configuration wizard, you can also just accept all of the defaults, except that in the security panel you must uncheck the "Modify Security Settings" checkbox (Figure 4). This is because starting with version 4.1.7, MySQL uses a new authentication algorithm that is not compatible with older client software, including the current version of Rails. By unchecking this box, you can access MySQL without a password.

This is not the path to secure computing. MySQL should NOT ship with a blank root password. Tutorials should not encourage the use of blank root passwords.

And they have you set up your server as to leave database.yml publicly available. I see Drupal attacks (xmlrpc.php) every day; it's only a matter of time before I start to see RoR attacks.

It's the developers' job to make it work. It's your job to make it work securely. Today's hackers don't even know C and have never heard of Kernighan and Ritchie; all they need is a script and an Internet connection to take advantage of your vulnerabilities.

Ruby on Rails and Security

Ruby on Rails is the most recently hyped language, so I though about testing it out on my development server. I followed the tutorial available on the RoR website. It went fine until I did a ./scripts/generate command and got lot of syntax errrors:

/usr/lib/ruby/1.8/yaml.rb:133:in `load': syntax error on line 27, col 2: `  host: localhost' (ArgumentError)
from /usr/lib/ruby/gems/1.8/gems/rails-1.1.2/lib/initializer.rb:459:in `database_configuration'
from /usr/lib/ruby/gems/1.8/gems/rails-1.1.2/lib/initializer.rb:181:in `initialize_database'
from /usr/lib/ruby/gems/1.8/gems/rails-1.1.2/lib/initializer.rb:84:in `process'
from /usr/lib/ruby/gems/1.8/gems/rails-1.1.2/lib/initializer.rb:42:in `run'
from ../config/../config/environment.rb:13
from /usr/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in `require'
from /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/dependencies.rb:147:in `require'
from /usr/lib/ruby/gems/1.8/gems/rails-1.1.2/lib/commands/generate.rb:1
from /usr/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in `require'
from /usr/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/ 

I started looking around at the folder structure that Ruby installs itself into when you create a new RoR application. Below my main folder, which the tutorial instructed my to create an alias or virtual directory for is the config folder. Inside the config folder is the database.yml file, holding my database information, with accounts and hard-coded passwords. (On my box, it's all localhost only, but still...)

Just to check, I fired up my browser and entered http://myserver/myrailsalias/config/database.yml. All the information popped up. I changed the Apache alias to /mypathtorails/public/ which I didn't see in the tutorial. This seems to be a lot more secure. This doesn't mean RoR is any more or less secure than any other interpreted scripting language for web applications, just that right now, it's easy to install it in a less secure manner.

What's the point? Know what you're installing, where it installs, what permissions it needs, and what context it runs as. And don't put your database.yml someplace where anyone can download it. I know there are websites where I could find it, but I'm not going to try. That doesn't mean someone else isn't writing a bot to find it right now.

Oh, and know how Apache works and httpd.conf works, too. All that is a lot to expect for people looking for a simple programming language.

DateDiff: How soon is now() ?

For a grad school project we need to build a system that bills people by the amount of time they rent a car. Thus, I needed to use the DateDiff function of VB and SQL in Access. I looked it up using Access help and got to the right MSDN page:

Syntax

DateDiff(interval, date1, date2[, firstdayofweek[, firstweekofyear]])

They tell you that interval is a String and link to what a string is. The part they don't say explicitly is that you need to enclose the interval value in quotation marks. "Of course," you say, "It's a string." Is it too much to ask for an example showing it in use with the quotation marks? e.g.

In Microsoft Access:
WHERE (((Rentals.CarOut)=True)) AND ((DateDiff("h",Rentals.DropOffDate,Now())>3));
In Microsoft SQL: (Transact-SQL for SQL 2000/2005)
WHERE (((Rentals.CarOut)=True)) AND ((DateDiff("h",Rentals.DropOffDate,GETDATE())>3));

Where h is the code for hours.

I just wanted to write this down someplace before I forget it. And why must Access and T-SQL be different? Now() doesn't work in T-SQL and GETDATE() doesn't work in Access.

On switching languages: things that make you go hmm.

When you switch languages a couple of times a day, you begin
to confuse some syntax and notice differences between the languages. In
one week: get basketball tournament
application working again. (PHP) Get Java app connected to Derby open-source database for my Java
class (handed in to my professor before class meets, but implemented in a
class). Get VB (6, not .net) apps
working for Decision Support Systems class (targeting zip codes) and Databases
class (rental car system). Get VBscript flight-risk scoring system
working. While Visual Basic and VB Script are close, they’re not the same. You
can dim intX as int in VB, but you can only dim intX in VB Script.  (The again, my server doesn’t even let me do option
explicit: The specified 'option
explicit ' option is unknown or invalid.)

And while you can do a Adodc.recordset.recordsize in VB6,
Java (1.5) has no resultSet.recordsize. So when I do a query in VB using ADO, I know how many
records I have; but in Java, I have to do while(recordset.next()) { recount++}
(source).
So my obsolete version of VB knows how many records are in my set, but the
latest version of Java does not. I’m sure there’s a reason for this, I just
don’t know what it is. I haven’t figured out if there’s a recordset size()
function/method in PHP yet. I wonder if Ruby on Rails has one available – I’d
check but the RoR API documentation
isn’t searchable.

Free Development Tools from Microsoft, Sun

Since many developers are writing free and open source tools for Linux, well, many free and open source tools are available for Linux. The cost of entry to development is low, and thus there are free applications competing with each other on Sourceforge. (E.g. Gallery, one of my favorites.) There are even free development tools and IDEs (Integrated Development Environments) for Linux.

 

Microsoft and Sun have now realized that having many developers working on your platforms could help sell them. Microsoft’s IDE, Visual Studio, has a free version, as well as a free version of its latest SQL server, SQL 2005. While it doesn’t have all the features of the full version, it’s pretty good, and will allow you to develop applications on Microsoft platforms.

 

Sun has also responded to the Linux development juggernaut, but has bet the farm. Sun has some of the most mature and detailed tools in the developer world. Until recently, they were expensive. Now you can download them free when you join the Sun Developer Network, which is also free, but you have to register. Sun has gone further than Microsoft, making nearly all of its operating system software free as well. Sun is also doing its best, as is Microsoft, to create a friendly space for developers.

 

Sun is betting the farm because, despite the quality of its products, Sun was about to become irrelevant in the face of the dot Net – Linux battle. Linux is free, but dot Net presents a friendlier interface and more standards. Sun’s tools were even more expensive than dot Net.

 

Whatever your choice for development, it’s a great time to be a developer. The Internet is getting interesting again. This year may be the one when the hype returns.

CS vs. IS vs. IT

Computer Science and Information Systems are fundamentally different. In Information Systems, we might be able to use a more efficient compression algorithm, but unless the business rules call for it, we don't care. IS is about management of business rules and processes, although some CS folks seem to think it's just a survey of computer science for people who can't do calculus. We actually end up doing calculus for project management to predict what our chances are of finishing a project on time. IS also covers organizational behavior.

I come from an Information Technology background. When I started talking about IT in an IS context, my professors quickly corrected me. Technology -- whatever it is -- should support business processes. Where technology can transform an organization is where IS fits in. Servers, routers, switches, and software are what you build IS implementations on. IS covers transaction support systems, management information systems, and decision support systems. IS is part of the business school; CS is part of the engineering school. IT is taught at vo-tech schools.

IS is why restaurants serve hamburgers with a pound of beef. By analyzing restaurant ordering system logs, it became apparent that giant burgers bring customers in and make more money for the restaurant chain. Computer scientists can write the DBMS that holds the data, and IT people can manage the hardware and the software, but the IS people design the system.

So when people start lumping everything together, IT/IS/CS, they are covering a lot of different areas. Some even talk about a degree in IT, which doesn't exist, at least not until Microsoft creates its own university. (Which McDonald's has.)

Object Oriented Development

Everybody keeps asking me what they teach in Information Systems grad school, so I'm going to start discussing it here.

Basically, we're still learning System Development Lifecycle, in several different forms. Structured, Rapid Application Development, Extreme (Xtreme!) Programming, and finally Object-Oriented.

Object Oriented is not just a higher-level language any more. It's a whole development method with its own techniques, tools and diagrams. These diagrams, more or less, replace dataflow diagrams and entity relationship diagrams, although I think every database should have a good ERD.

Some of my classmates were still confused and/or think OO development is a fad. It slowed class up a little. A fair misunderstanding is the database side. What's an object oriented database? Well, we're not really sure yet. OO programming still uses databases -- traditional databases. From the diagrams and textbook, this wasn't clear, leading to the question, "Where does the data go if you turn the system off?"

It's still there, in a database, just like structured approach.