I publish my own weather station data on the web

I finally got my weather station published to the web. Thanks to wview and Davis, you can see the temperature at my house. The install went fairly easily, with just one non-rpm'ed/yum prerequisite (radlib) requiring an install from source, along with wview. The first install didn't connect to sqlite correctly, but I fixed that with a fresh install and skipping wview's Davis-station specific setup. I am also considering migrating to Debian, but that's another topic.

I love the data views that wview provides. It shows trends at a glance that weather.com, NOAA, and NWS don't. Even online weather airport stations have no view that will let you see a trend -- they're all snapshots. With wview, I can tell when the temperature-dewpoint spread is closing.

The best part -- I can collect the data. I can export to MySQL. I can collect solar radiation stats to see if solar panels make sense at my location.

Next: Dial-in weather with Asterisk and Cepstral.

Top 5 things to do with cheap Intel Atom servers and Free Software

Intel introduced the low-power, affordable Atom chipset a couple years ago. The earlier models included the Atom 230 and 330, running single and dual-cores, respectively, at 1.6 GHz. Frugal with power at 8 watts, it could even run 64-bit. The first-gen Atom motherboards/chipset kits had two chief weaknesses: they maxed out at 2 GB of RAM, and they used RealTec network-interface cards (at least the Supermicro mboards did). RealTecs sometimes crash Linux when the incorrect driver loads, and the RT NICs don't do jumbo frames. The crashes are patchable but it adds a couple hours to your install. (Do not, repeat not, buy the old models no matter how cheap they are. Stick to the D510 series.)

Intel has since released the Atom D510. The processor is up to a 13-watt draw, runs at about the same speed, and still runs 64-bit. What makes these chipsets better is that they max out at 4 GB of RAM and use genuine Intel Gig NICs that are widely supported and can run jumbo frames. This is a big leap, because the configured price of a fully-built mini-server around the platform remains the same. Supermicro has some great Atom motherboards that can support a home or small office doing various things. (The D525 has since come out. Compare the two here.)

Newegg has some good deals on the Supermicro Atom Rack Server and the tower server for about $70 less. Or you can just buy a motherboard and roll your own. My tower server with D510 runs just under 40 watts with 4 GB of RAM, CD, and hard disk.

What can you do with it? There are a number of Linux and BSD appliances that are free.

1) Run your own telephone switch. FreePBX is an awesome, OS-included version of Asterisk that has a clean web GUI. PBX-In-a-flash is based on FreePBX but has a few extras (nonstandard enhancements like dial weather by airport code and dial your local tide schedule) thrown in. Why would you want to run your own PBX? You can do stupid phone tricks, like really cheap phone service at $.01/minute and $1.50/month per number with Vitelity, for one. Or your own conference line. Or your own trunks over VPNs to other offices. Or just blacklist phone numbers and send telemarketers to their own voice-mail hell.

Keep in mind that to light up your home phone lines, you may need an additional analog card from Sangoma or Digium. (FXS and FXO are different, but can run on the same card.) Once you figure out how cool SIP is, you'll want to replace your house phones with SIP phones. However, you can get started with SIP free using the Xlite soft phone. The iPhone/iPod/iPad version is $7.99. It's perfectly true that you can make phone calls cheap with a Linksys PAP2T or Vonage, but if you're a telephone control freak and want to wage war on telemarketers, Asterisk is for you.

2) Run a great firewall like pfSense. It's based on FreeBSD but has a great Web GUI so you can avoid learning the subtle differences between BSD and Linux. pfSense comes in embedded and full. For the Atom server, running the full version is no sweat. And I run SNORT inline on two interfaces with no issues. A full SNORT IPS signature subscription for home use is just $30/year. And pfSense does not just IPSEC point-to-point VPNs, but also WebVPN. Try that on your Cisco ASA 5505, and you'll be paying a couple grand for the equipment, licenses, and subscription. Check out the plugins for pfSense and you'll see how much you can do with one small box. pfSense 2.0 is almost out of beta and there's a 64-bit version. (Plugins include FreeSwitch, an open-source competitor to (open-source) Asterisk, so if you want, you can even run a PBX on your firewall.)

3) Storage: Run FreeNAS and you'll be able to scale reasonably out of the box. The Atom Motherboard has six SATA slots and two GB NICs that do jumbo frames. Hello iSCSI. With 2TB disks, that's 12 TB of storage. FreeNAS offers CIFS (Windows), Apple (AFP), iSCSI, and NFS mount points. Add a serious SATA RAID card to get even more disk IO performance out of your little Atom box. Obviously, I would recommend the tower server case for FreeNAS because it has room for more disks.

4) Run Apache/PHP/MySQL applications. Run Fedora, CentOS, whatever. You can run your own apps at home on top of your favorite flavor of Linux/BSD. One of my favorites is Gallery2. There's even semi-free apps (free for no enterprise features) like Movable Type (you're reading this on an MT platform), and even run more than one on the same mini-Server.

5) Collect data. Run a weather station. Run a log server. Run a crazy blog and see who hits it with which keywords. Collect IPS events from multiple firewalls and correlate them.

Migrating Movable Type and Gallery2 to a new (Fedora) server

I loved my Dell 1750 server. It has plenty of power and a great 3Ware RAID card, two 500 GB RAID-1 drives, 4 GB RAM, and ran Red Hat Linux. I bought it from the Dell Outlet site several years ago when my condo fee included electric. Since then I have moved. I pay my own electric bill, and my 1750 consumes 150 watts at idle. When I publish with Movable Type or Gallery, power consumption exceeds 200 watts. I pay $0.150845894 per kilowatt-hour. (That's summing the separate generation, transmission, distribution, demand-side surcharge, and adding the gross-receipts tax. Pepco doens't make it easy to figure out what you're paying.) With a thirty-day month, that's 108 kwh, which comes to $16.29135658. That's $16/month, just for idling. And that doesn't use the noise of the server in my office or the additional AC required in summer.


That may seem expensive, but it's far cheaper than getting that much server capacity at Rackspace. However, I don't need that much capacity. I can trade processor power for power savings, keep the disk space and RAID card, and switch to an Atom-based server. My current FreePBX Atom server runs at 40 watts with an analog card powering two FXS modules. I bet I can match that on a new server. I'd get the new Supermicro Atom Server, but it has space for only one 3.5 inch hard drive. I need two. Thus I'll be using another miniITX case. In the meantime, everything's running on another old Dell tower box.


How did I move it? First I though reinstalling all the software from scratch would be a good idea. I'd get a nice clean, efficient build. But that took way too long, and I'd have to re-customize my templates and tweaks. I had MySQL backups running for a while, why not start testing the restores? I used rsync:

rsync -avz /var/www/ -e ssh:user@mynewserver /var/www/

(Note: Please study rsync syntax. Those / at the ends make a big difference.)
It worked. Next I had to tweak the new httpd.conf file. I couldn't just copy the old one, because I was using the latest Apache version. But I could use almost all of the old file. I just needed to adjust the modules it loaded, because several have changed names.


Then I restored the databases:

mysql -u root -p

enter your password. (You ARE using a PW for MySQL root, aren't you?)

mysql> create database mynewdb

then

mysql> quit;

then

$ mysql -u root -p [mynewdb] < [backupfile.sql]

But that generally does not restore your user privs on the db. Back to mysql:

mysql -u root -p

then

mysql> use mynewdb;



mysql> GRANT ALL PRIVILEGES ON *.* TO 'myuser'@'localhost' 



IDENTIFIED BY 'pAssW0rd' WITH GRANT OPTION;



mysql> flush privileges;

Don't forget the above step or you'll need to restart MySQL to get it to work.

mysql> quit;

Then double check that the user and pass from above match your config files.


Finally, test your applications. Gallery2 and MovableType worked fine. Your milage may vary. My office is almost silent now.

How to tell when someone Googles you

Case 1: You Google me and click on my page

Yes, I'm using google as a verb. If you Google me and click on one of my pages, my web server logs the information:
1.2.3.4 - - [01/Oct/2009:10:23:41 -0400] "GET / HTTP/1.1" 200 7186 "http://www.google.com/search?hl=en&source=hp&q=larry+s&aq=f&aqi=g10&oq=&fp=7d15299a959dbb33" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)"

As you can see, I get your IP address, a date, an offset to Universal Time (-0400), a verb (GET, in this case / means my default site page), a status code (200=OK), and a referrer. From the referrer, I can tell you Googled me with the phrase "larry s". Finally, I also get some information about the browser you used, Firefox, and the operating system, Windows XP with service pack 2. There's a chance you may have used a anonymizing proxy, but I'd still get an entry. (Generally, Anonymizer says "TuringOS," so I know it's them.)

Case 2: You Google me and don't click on my page.

That's more difficult but not impossible, because I have a Google AdWords account. I bought my own name as a keyword. Google AdWords works by selling keywords for search insertion. It's an open market, with the second-highest bidder winning in a dutch auction that is Google's revenue machine. When you buy a keyword, you get two measures back from Google:

1. how many impressions it got (viewing)


2. how many clickthroughs it got. (someone clicks on the ad)

A keyword ad's success is measured by the ratio of impressions to clickthroughs. The more clickthroughs per impression, the better. So if you don't click on my ad link, which I have made irresistable by promising dirt on me, I still know that someone Googled me, because the impression counter increments with each search. If you click on a regular page on my server rather than the keyword ad (Google calls this "organic"), we're also back to case one above. If you don't click on any of my links, I don't get any of the details from case one.


And that's how I know that you Googled me. If you're wondering if you've been Googled, but don't have a web site with logs you can comb through, or don't want to set up a Google AdWords account, try Google's external keyword tool. Just don't forget to un-check the synonyms box.

Online Backups with Backblaze: Does it work?

Just because disk space is getting cheap, don't think that storage is cheap. A referral from Slashdot to Backblaze's blog charted the situation out accurately. The cost of a petabyte of storage on raw SATA hard drives is $81,000. On Amazon and EMC, it's $2.8 million. If Backblaze really could create their own enterprise storage devices, then it would be possible to offer backups at $5/month for unlimited storage. Backblaze even offers a 15-day free trial, so I tried it, although I was skeptical.

Catches:
1) You need to use their client.
2) Their client doesn't run on Windows Server or Linux -- just WinXP, Vista, and Macintosh. (Even if you run the installer in XP compatible mode on Windows server, it still doesn't install.)
3) The $5/month is for one computer, not all the computers in my house.
4) rsync doens't work with Windows/samba shares. (You may, however, be able to get rsync to work to a Macintosh. I haven't tested yet.) (Update below: you can install an NFS server onto WindowsXP/Vista to get rsync to work, or you could do it from Windows via an SSH rsync script.)

Solution: I installed it on a Vista workstation, created a share, and copied the few things I really need backed up to it. I also wrote scripts to transfer my PBX backups to my backup and log host and then copy the files from the backup server to the windows share via smbclient. I'll skip the part about configuring password-less logins for SSH via ssh-keygen keys, as well as the kinit for logging into windows via smbclient. (I also never was able to mount.cifs via kinit, just smbclient.)

So what happened when I tried to back up 15 GB on my Vista box to Backblaze? Not much -- the files just transferred. iPod library -- check. Photos -- check. My mrtg indicated that bandwidth increased to about 310 kbps for four days. I was still able to make phone calls via my SIP trunk to vitelity with no problems. (g729 to my SIP provider and alaw to my friends' PBX servers via IPSec VPN.)

Bandwidth used:
`Weekly' Graph (30 Minute Average)


Max Average Current
In 501.9 kb/s (0.5%) 56.4 kb/s (0.1%) 47.9 kb/s (0.0%)
Out 1360.2 kb/s (1.4%) 168.6 kb/s (0.2%) 29.3 kb/s (0.0%)

Security comments: Backblaze says it encrypts files, but doesn't offer details on the algorithm or implementation. (e.g AES-CBC, etc.) Backblaze does offer you the option of using a private key, so that only you (assuming you don't forget the key) can access your files.

My advice: If it needs to stay secure, encrypt the files yourself before they hit the local hard disk. You can even do a loopback mount (Super-awesome tutorial there) to an AES-encrypted file on a samba share, and rsync will work, but the whole file will change, requiring it all to be sent to Backblaze.

Update: You can use rsync to get your Linux/BSD/Unix files over to your windows box, but you'll need to install an NFS Server on your windows box. You could also use Microsoft's Services for Unix, but it's easier just using the Allegro server.

Cisco MIB: Interfaces on the 3845 Router

Recently I needed to check traffic on specific interfaces of a Cisco 3845 Router. I didn't have a MIB file uploaded to our SNMP workstation, and descriptions of measures were not in synch with the router. Thus I needed to figure out which interface was which. There were 8 valid instances of interface metrics on the router. I was interested in BitsIn/Sec, BitsOut/Sec, and IntSpeed. From IntSpeed, I got the following numbers:
1. 1,000,000,000
2. 1,000,000,000
3. 4,294,967,295
4. 44,736,000
5. 45,000,000
6. 44,736,000
7. 45,000,000
8. 4,294,967,295

Thus I figured out that Serial 0 is 5 and serial 1 is 7. Gig 0 and Gig 1 are 1 and 2. We have two DS-3 circuits (ATT calls them DNECs) in. SNMP may be wonderful but MIBs are a pain. I thought I would write this down before I erase my whiteboard with tomorrow's problem and solution. You can find Cisco's guide to it's MIB and SNMP here.

The Dump Cheney Rumor

Anyone reading Drudge today will notice that there is a report that Bush will dump Cheney
following the 2006 elections. Those people taking this report at face
value would do well to remember what Bush's father, George H.W. Bush,
did in 1992 with his own Vice President, Dan Quayle.


Like Cheney, Quayle had a few liabilities with the press corps (Murphy Brown, e.g.), and his
Chief of Staff, William Kristol, was determined to prop his boss up,
like any good CoS. Thus, Kristol leaked a story that Bush would replace
Quayle. The press corps forced Bush to issue a denial, solidifying
Quayle's number two spot on the GOP ticket.




Dumping Cheney after the elections won't make any difference for this
year's results, which is what Republicans worry most about.  Next
year is Bush's last term in office, and installing a new Vice President
isn't that easy when you consider the vetting process and the Harriet
Myers withdrawal.



Historical moments in Dick Cheney's life: His opinion of Adam Clymer.

HDTV Page Updated

I just checked the logs for my site, and while more people
continue to look at this blog, more are looking for HDTV information in
Washington, DC. I think a lot of people are trying to figure out why
they're not seeing the Olympics in HD on their new HD sets. Thus I have updated the page.

The Neurochemistry of Schadenfreude

Update, 09-25-2009: A postdoc has proven that dead salmon could apparently detect, and respond to, the the emotional state of human beings, according to his fMRI research. That's better than the fish sticks joke.

After reading Slashdot's latest on
lie-detection using an MRI (Magnetic
Resonance Imagery), I read a submission to Nature on "Empathic
neural responses are modulated by the perceived fairness of others," which
also used MRI to detect emotional states of its subjects. (The title is from
the cover of Nature's 26/01/06 issue.)  Basically,
a team from CalTech,
the Wellcome Department of Imaging,
Neuroscience, and the Institute
of Cognitive Neuroscience at University College of London, used the
Prisoner's Dilemma game to induce liking and disliking of opponent players. Emotional
reactions could be located in specific regions of the brain. 

Being able to "see" emotions in the brain sounds pretty
cool, but these emotions have been known to be "real" since Homer told the
story of Apollo helping Paris
kill Achilles. The new technology for lie detection sounds like it could be
used for great profit, but how would you feel being asked for a "lifestyle MRI?"
The lifestyle polygraph is rather feared because the more conscientious you
are, the more likely it is to trip you up. (Remember that pen you "stole" by
failing to return it?) However, the polygraph is inaccurate and hasn't caught
any spies yet. (They'll say it helped, but it didn't stop Aldrich Ames or
Robert Hansen.)

Who will administer the lie-detection MRI? It's not like you
can go get a six-week degree in radiology and neurology, as you can for the polygraph. It takes a board-certified
radiologist
just to find something actually wrong with you in an MRI, not
just a subtle change in a specific area during the subject's response.
I doubt
highly that if any companies start setting this up that they'll even
use
scientific staff to run it. I doubt that many doctors would even
testify to its
reliability, but that won't stop it from being used for profit.
Fortunately, the courts have refused to admit polygraphs as evidence
and will do the same for MRI lie detection.

Someone Used My Photo

A blogger linked to my photo from her page. I'm flattered. I thought it was almost a throwaway photo, but posted it anyway just because I liked the view of Times Square from a New York Taxi. Since she didn't copy it outright, I could tell from the referrals to my site and I added a credit to the photo. I still have no idea how she found it.

The depressing part is how totally un-hip and geeky my blog is. But that's me.

original photo

Red Hat vs. Mandrake

I grew tired of updating every package that came with Mandrake. The
most frustrating part was not finding what I needed to install, or
installing it but then configure didn't find it.




I re-downloaded Fedora Core 4 in the DVD image, burned it on the Mac.
Macs can open, create and burn ISO images using the standard Disk Tools
utility. I checked the sha1sum on the Linux box that was the FTP
staging area, because OS X doesn't seem to have a sha1sum utility, and
I didn't feel like spending time downloading, configuring and
installing one. CuteFTP balked at downloading a 2.6 GB file, too,
insisting that there wasn't space on my hard disk for it, even though
there was plenty of space. Once again, Linux command line to the rescue.




Red Hat is now installed on my antique Inspiron 7500, and it's not
perfect yet -- I'm still working on the display. But much more software
works without endless downloads configure-make-make install cycles....

Progress

Back in the old days, (1990s) I had ISDN at home. It was the fastest bandwidth available, and it afforded me voice and fax and data all in one interface. (ISDN-BRI, with two B channels and one D channel for signalling, if you get technical.) Billing was awful. I paid two cents a minute per B channel during peak hours, and it took some tweaking to get it down. (Windows really does broadcast every eleven minutes, and that would bring up the data at two cents a minute.) Needless to say, I stayed up late at night to do large downloads.

I have gone through DSL and now I have cable at about 5 Megabits per second. I just downloaded Fedora Core 4 (Redhat's free testbed) and installed it on an old P3 system I have here. Federal Core 4 is 4 CDs, about 650 MB each. I had some trouble with the media checking, and the SHA-1 checks didn't check out, so I had to download them all again.

All in all, I think I downloaded over 5 GB of data over the past 24 hours. Back in 1998, this would have been unthinkable. Back then, you had to order the CDs, wait for them to arrive, and then install it. And installing Redhat 5.x, you had to know the IRQs and DMAs and which chip set your NIC had (I started hoarding DEC Tulip cards). Today, Redhat (and the other Linuxes) load up all my hardware automatically. I don't have to know anything about my hardware.

And the new linuxes have nice GUIs that launch by default. To log in at runlevel 3, you need to start tweak inittab. And VI has now been replaced with VIM.

All I wanted was a command line interface like I'm used to.