cmb's tech blog

Sniping eBay Auctions with esniper on FreeBSD

2006-03-20T13:04:00.000-05:00

I've been using esniper to automatically place bids on eBay for a couple months now, and it's worked flawlessly. Check the esniper web site for information on what sniping is, amongst other related questions. They explain that well, I won't duplicate their explanation here.

So what does esniper do?
In a nutshell, esniper allows you to bid your maximum amount at the last second, either on a single item, or on a list of items. I use it for lists as much as anything, because I'm frequently looking for multiple identical items, like 5 of switch model X, or 7 of router model Y, etc. I can define a quantity in the auction file, then put in a list of auction numbers and my max price, and it will automatically bid until I have won the desired quantity (or it reaches the end of my list of items).

Installing esniper on FreeBSD
esniper can be found in the FreeBSD ports collection under /usr/ports/misc/esniper. To install:

cd /usr/ports/misc/esniper && make install clean

Using esniper on FreeBSD
First, you need to create a .esniper file in your home directory (note the leading dot) containing some configuration information.

Here is the ~/.esniper file I use:

# A configuration file contains option settings,
# blank lines, and comments (begin with #). The
# default configuration file is $HOME/.esniper, and
# if you specify an auction file, .esniper in the
# auction file's directory as well. If you specify a
# configuration file using the -c command-line option,
# then that configuration file will be the only one
# read.
#
# If you set your password in a configuration file
# and you are on a unix system, you should be the
# owner of the file, and you should set the
# permissions so that it is readable only by the
# owner.
#
# Here are all the options. Note that if the
# default (as defined in esniper -H) is fine, you
# don't need to specify it in a configuration file.

# boolean parameters can be set to:
# true, y, yes, on, 1, enabled
# false, n, no, off, 0, disabled
batch = false
bid = yes
debug = enabled
reduce = y
username = EBAYUSERNAME
password = EBAYPASSWORD
logdir=~/esniper/logs

# Note: quantity is usually set in an auction file
quantity = 1
# Note: seconds can also be "now", if you want to
# bid immediately
seconds = 10

Replace EBAYUSERNAME with your eBay user name, and EBAYPASSWORD with your eBay password. The quantity above is the default quantity, and seconds is how long before the end of the auction that esniper will put in your bid by default. You can change "bid = no" if you just want to see what it would do, rather than letting it start bidding.

I create ~/esniper/ and ~/esniper/logs/ directories, and put auction files in ~/esniper/ and the above .esniper file sends logs to ~/esniper/logs/. Make sure you 'chmod 600 ~/.esniper' after you create the file, to help protect it from other users since it contains your plain text eBay password.

Also check the sample configuration from the esniper web site.

Creating an Auction File
For each type of item you wish to bid on, you need to first create an auction file. I call it something that makes sense to me, based upon the items within it. You can call it whatever you like.

You define the quantity of items you want, the auction numbers you want to bid on, and the dollar amount of the bid to place.

See the sample auction file on the esniper website for details.

Starting esniper
Once you have your configuration file and auction file created, you can run esniper. To run it using the auction file "my-auction-file.txt" in ~/esniper/:

$ esniper ~/esniper/my-router-auction-file.txt

And let it run, it will check with eBay periodically and place your bid at the time you defined in your .esniper file.

Have fun!

FreeBSD 6.0 and VMware 5.5

2005-12-20T01:47:00.000-05:00

FreeBSD has always worked well "out of the box" for me in VMware, until FreeBSD 6.0 and VMware Workstation 5.5. FreeBSD was having serious issues with time, which caused serious issues with the network stack, amongst other things.

Thanks to some advice from Scott Ullrich, and a VMware forum post from Richard Bejtlich, I was able to get things going.

In short, add the following two lines to /boot/loader.conf

hint.apic.0.disabled="1"
kern.hz="100"

And one line to /etc/sysctl.conf

kern.timecounter.hardware=i8254

Then reboot, and all will be well.

Upgrading FreeBSD 5.4 to 6.0

2005-10-16T20:14:00.000-04:00

I upgraded my first production box from FreeBSD 5.4 to 6.0 with great success. This box runs Apache, MySQL, PHP, and my ncid server. Other than ncid, nothing else is really production nor critical on this box. This is my least "production" server so it was the least-risky to attempt.

I did the usual cvsup, using tag RELENG_6 to get the 6.0 RC1 sources, and built and installed world/kernel and ran mergemaster. For more details on how to do these things, I recommend Richard Bejtlich's excellent Keeping FreeBSD Up To Date article. Richard does a good job of addressing everything there, no need to reinvent the wheel.

Everything came out perfectly well with one small change with serial port naming that kept my ncid server from starting. Serial ports in 5.x were /dev/cuaaX and are now /dev/cuadX where X is the number of the serial port. This box has one serial port so it changed from /dev/cuaa0 to /dev/cuad0. A simple change in my ncidd.conf file and manually firing up ncidd fixed that.

All else is running well with no other changes at all. Looks like FreeBSD 5.x to 6.0 upgrades will be pretty painless.

Installing MySQL on FreeBSD

2005-10-01T21:34:00.000-04:00

I'm installing MySQL in a fresh FreeBSD jail and thought it'd be a good idea to document the quick, easy process, so the next time someone asks me how to do this, I can just point them to this.

Update your Ports
First, cvsup your ports collection so you have the most recent versions available.

Choose a Version
This isn't very clear or simple. There are four different MySQL versions in FreeBSD ports - 3.23, 4.0, 4.1, and 5.0. Unless your intended application only works with MySQL 3.x, I wouldn't recommend using that version. MySQL 5 is the current development version, so I wouldn't recommend it for production use either. MySQL 4.1 is the latest "production release" as of this writing, so generally this is what you will want to use.

Installation
The installation is almost as straight forward as usual. There are some make options that may be of interest.

You may use the following build options:

WITH_CHARSET=charset Define the primary built-in charset (latin1).

WITH_XCHARSET=list Define other built-in charsets (may be 'all').

WITH_COLLATION=collate Define default collation (latin1_swedish_ci).

WITH_OPENSSL=yes Enable secure connections.

WITH_LINUXTHREADS=yes Use the linuxthreads pthread library.

WITH_PROC_SCOPE_PTH=yes Use process scope threads
(try it if you use libpthread).

BUILD_OPTIMIZED=yes Enable compiler optimizations
(use it if you need speed).

BUILD_STATIC=yes Build a static version of mysqld.
(use it if you need even more speed).

WITHOUT_INNODB=yes Disable support for InnoDB table handler.

WITH_NDB=yes Enable support for NDB Cluster.

For the most common uses you can get by without any options. I typically use BUILD_OPTIMIZED and BUILD_STATIC.

cd /usr/ports/databases/mysql41-server
make BUILD_OPTIMIZED=yes BUILD_STATIC=yes install clean

MySQL will now install.

Configure to Start on Boot
A startup script is installed in the usual location, /usr/local/etc/rc.d/. Before this startup script will function, you need to open your /etc/rc.conf and add the line mysql_enable="YES".

Start MySQL
MySQL is now configured to start at boot, but you probably don't want to reboot just to get it to start. Just run the following.

/usr/local/etc/rc.d/mysql-server.sh start

Now MySQL is running and ready for use.

Recent Happenings

2005-10-01T21:18:00.000-04:00

Couple things keeping me busy lately.

One, the new m0n0wall Documentation Site and all the new documentation contained therein for this open source firewall project. This makes public much of the work I've done over the last several months, and will become the new location for documentation as soon as I get it in a bit better shape. I'm hosting this site on my colocated server at Louisville ISP bluegrass.net's data center, rather than on the main m0n0.ch site. This makes it easier on Manuel, founder of m0n0wall, and I, since I write most of the documentation and he isn't involved much in it. The documentation for open source firewall project pfSense will begin to be forked off of this site once we decide on a suitable documentation platform.

Second, I've been working with best selling computer book author Dave Taylor as coauthor on a series of e-books. Connect Safely is the name of this project, offering Wireless Security and Privacy Guides. My typical audience isn't really the intended audience for this series, but if you're a frequent wireless network user and aren't extremely familiar with wireless network security concerns, you really should consider this. I can't tell you how many even quite computer literate folks I've seen giving their passwords to the world on a hot spot. As a bonus for more technical readers, I wrote a section on using Ethereal to analyze your network traffic to ensure it is being protected. Stay safe on wireless networks - check it out.

Replacing Notepad on Windows

2005-07-19T14:35:00.000-04:00

A fast lightweight text editor is always nice to have, but on Windows, Notepad falls far short. There are a few light text editors for Windows that are just as light and fast as Notepad, but include features that frequently come in handy.

Metapad is my Notepad replacement of choice. A few notable features it offers that Notepad lacks follows (with a full list on the Metapad site).

URL's are clickable links
Doesn't botch up non-Windows text files (some Unix and Mac file types are problematic in Notepad)
Line and column count
No file size limit

Actually getting Notepad replaced is a major pain. File associations, context menu replacements, etc. etc. It's much easier to just replace notepad.exe. But Microsoft makes that difficult too!

notepad.exe is located in C:\Windows\system32. But if you delete it, or rename it, as soon as you do Windows will replace it. This is a "feature" of Windows File Protection. When you do this, you'll see a message like the following in your Event Viewer:

Event Type: Information
Event Source: Windows File Protection
Event Category: None
Event ID: 64002
Date: 7/19/2005
Time: 5:06:58 PM
User: N/A
Computer: CMB
Description:
File replacement was attempted on the protected system file notepad.exe. This file was restored to the original version to maintain system stability. The file version of the system file is 5.1.2600.2180.

To get around this, I run a batch file with the following three lines. The batch file lives in the same directory as the downloaded metapad.exe.

copy metapad.exe %windir%\system32\dllcache\notepad.exe

copy metapad.exe %windir%\system32\notepad.exe

copy metapad.exe %windir%\notepad.exe

The dllcache directory is where Windows File Protection keeps its files used to replace protected files if they become modified. Windows obviously doesn't protect the dllcache the way it does the rest of the files, so the above seems to work fine (at least for me). I've used it on Windows 2000 Server, XP Pro, and Server 2003 without a hitch.

Gathering Cisco Router Configurations with Perl

2005-06-14T14:23:00.000-04:00

Have a number of routers from which to gather configurations? The following bit of Perl makes this extremely quick and easy, if they all have matching telnet and enable passwords. Must install the CPAN module Net::Telnet::Cisco, and enter the IP addresses of the routers in a file called routerlist.txt, one IP per line. To use this, you just need to replace "TELNET_PASSWORD" and "ENABLE_PASSWORD" below with the appropriate real passwords from the router.

It'll output files in the directory you run it from called IP.add.re.ss.conf, one file per IP.

#!/usr/bin/perl -w

use Net::Telnet::Cisco;
@routers = `cat routerlist.txt`;

foreach my $router (@routers) {
      chomp($router);
      my $session = Net::Telnet::Cisco->new(Host => $router);
      $session->login('','TELNET_PASSWORD');
      $session->enable("ENABLE_PASSWORD");
      @conf = $session->cmd("wr t");
      open LOG, "> $router.conf";
      select LOG;
      print @conf;
      close LOG;
}

Disclaimer: Are there better ways to write this? I'm sure there are. I don't proclaim to be a programmer, but the way I do things *works*, whether or not it's the "best" way. There's more than one way to do it!

Configuring Sendmail for send-only on FreeBSD

2005-05-25T14:11:00.000-04:00

One of the first things you'll want to do after a fresh install of FreeBSD is configure email so you get the daily run output and other various emails from the system. Assuming this system will not become a mail server itself, you can follow these steps to accomplish this.

Setting up Sendmail
First, in rc.conf, set sendmail_enable="NO". This configures sendmail for receive only.

Next copy the default freebsd.mc file to `hostname`.mc (where `hostname` will output the real hostname of the machine).

cd /etc/mail
cp freebsd.mc `hostname`.mc

Now open up `hostname`.mc in your text editor of choice, and go down to the following line.

dnl define(`SMART_HOST', `your.isp.mail.server')

Change that line to the following, replacing mail.example.com with the name of your or your ISP's mail server.

define(`SMART_HOST', `mail.example.com')

Save the changes to `hostname`.mc and exit your editor. Then run the following commands.

make
make install
make restart-mta

Redirecting root's email
Now open up your /etc/aliases file in your text editor, so you can define an email address to send root's emails. Go down to the following line.

# root: me@my.domain

Uncomment it, and change me@my.domain to your actual email address. For example, if your email address is me@example.com you would make the line look like the following.

root: me@example.com

Then save your changes and exit your editor.

Last, run the command newaliases to rebuild your mail aliases file, and you're set.

Alternatively, if you can get away with only text email functionality, ssmtp in ports is an easy solution. If you run a web site that needs to send HTML emails, as one example, this won't suffice since it will send them out in plain text. The FreeBSD Handbook documents how to configure ssmtp.

Ditch Task Manager - Process Explorer to the rescue

2005-04-29T01:57:00.000-04:00

Sysinternals has been a great resource for Windows tools and utilities that I've been using for a number of years.

Lately Mark Russinovich has been blogging about the many cool and useful things you can do with Process Explorer.

I've been using Process Explorer in lieu of Task Manager for a while now, and there's really no comparison. Give it a shot, and like me, you probably won't turn back to Task Manager.

The feature I use the most is searching handles. My most common use of this has been to resolve the dreaded "this file is in use" message when a process holds a file or directory open and it's not clear which process it is. Mark has described this process and several others on his blog, so I won't attempt to explain anything since he does a far better job than I could.

Maintaining Exchange SMTP Addresses with Recipient Policies

2005-04-24T23:29:00.000-04:00

Most of the time when I'm setting up a network, or upgrading a server, the Exchange email addresses differ from the Windows user name. Like the usernames might be FirstnameLastinitial, while the email addresses are FirstinitialLastname@example.com. You can use recipient policies in Exchange to automatically maintain the correct email addresses for all your users.

You can add a new recipient policy, or edit the default. For example, to use FirstinitialLastname@example.com, you can edit your default recipient policy to change the default "@example.com" to "%1g%s@example.com" (the first letter of the given name, and the entire surname).

How to Modify an SMTP E-Mail Address by Using Recipient Policies - Microsoft KB article

FreeBSD NDISulator

2005-04-17T01:02:00.000-04:00

FreeBSD 5 has introduced a NDIS emulator that allows you to use Windows binary drivers on FreeBSD, the so-called NDISulator, or "Project Evil". The main use for this is for wireless network cards because many vendors do not release source code or schematics for their drivers to the open source community. See section 25.3.3.6.3 on this page of the FreeBSD Handbook for documentation on using the NDISulator.

I have a Linksys WPC11 v4 PCMCIA wireless card. Versions 1-3 were supported by the wi driver in FreeBSD, but v4 is a Realtek 8180 chipset that isn't yet supported. (I won't get started on my soap box about how vendors should never change chipsets without changing model numbers) I do have a few supported wireless cards, but figured I'd give the NDISulator a shot with it. This is actually the card I regularly use in my FreeBSD laptop now.

Installation

Download the Windows XP 8180 drivers from Realtek here.
Unzip the downloaded file to:
```
/usr/src/sys/modules/if_ndis
```

Run the following commands:

# cd /usr/src/sys/modules/if_ndis

# ndiscvt -i NET8180.INF -s rtl8180.sys -o ndis_driver_data.h
# make && make install

The driver is now ready. To load it, run the following commands:
```
# kldload ndis
# kldload if_ndis
```

Now check your dmesg output.  If all went well, you'll see something like the following: 
ndis0:  port 0x1000-0x10ff mem 0x88000000-0x880001ff irq 11 at device 0.0 on cardbus1

ndis0: NDIS API version: 5.1

ndis0: Ethernet address: 00:0c:41:a7:98:45

ndis0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps

If all went well, you can now ifconfig the interface as desired (see the Handbook) and you're ready to go.

Performance
I initially had some serious performance issues. Throughput was inconsistent, and didn't exceed 200-300 Kbps. This was with a driver I found recommended on a FreeBSD list post or website I found on Google (don't recall where). After downloading the latest driver from Realtek, I'm getting good throughput. iperf consistently gets around 5 Mbps across the wireless connection, about as good as you're going to get on a 802.11b card.

Conclusion
Overall I am very impressed with the FreeBSD NDISulator. I wasn't expecting it to be as seamless, easy to get working, and well-performing as it is. Great work to Bill Paul and the FreeBSD team.

Setting up a DocBook build environment on FreeBSD

2005-04-10T21:16:00.000-04:00

Along the same lines as my previous post on setting up OS X to build DocBook sources, here is how to set up FreeBSD to do the same.

Just need two ports, textproc/libxslt and textproc/docbook-xsl.

cd /usr/ports/textproc/libxslt/ && make install clean
cd /usr/ports/textproc/docbook-xsl/ && make install \ clean

Then you can use a script similar to mine to do the build.

Setting up a DocBook build environment on OS X

2005-04-10T21:00:00.000-04:00

In my work with m0n0wall and pfSense, I work with DocBook XML sources frequently. I currently run my builds on a Pentium 4 1.6 GHz FreeBSD 5.3 box, but figured it would be easy to set up on OS X and should run quite a bit faster on my Mac. I was correct on one of those two.

Prerequisite Components
You need to install three things from Darwin ports before doing a build.

port install docbook-xml
port install docbook-xsl
port install libxslt

Then you just need a script to build the sources. The script I use is available here.

Speed Comparison
I posted earlier that StatCVS is dramatically faster on my 1.42 GHz G4 as compared to a FreeBSD 5.3 Pentium 4 1.6 GHz system. I was shocked to see that xsltproc is actually a decent amount slower on my Mac than it is on my P4 1.6 when building the m0n0wall Users Guide. FreeBSD takes 39 seconds, the Mac takes 58 seconds. These results are repeatable, +/- 3 seconds on each. It's CPU-bound, with xsltproc taking up 100% CPU while running on both.

If anyone knows why this is the case, or how to improve the performance on the Mac, please email me.

ext CVS on OS X through SSH

2005-04-09T13:43:00.000-04:00

By default, OS X tries to use RSH for ext CVS access, unlike other *nix-based systems I've used. If you're seeing "Connection refused" when trying to use ext CVS, this is probably why. tcpdump showed it was trying to connect to TCP 514 on the CVS server, which wasn't listening since it's running through SSH.

To fix this, edit .profile in your home directory (create it if it doesn't exist) and put in the following:
export CVS_RSH="ssh"

Save and exit, then log off and back on (or close your terminal window and open a new one) and things will work as you expect them to.

Thanks to Scott Ullrich for a clue to figure this one out.

StatCVS on Mac OS X

2005-04-08T15:36:00.000-04:00

StatCVS is a slick CVS statistics generation utility. Previously I ran this from cron on a FreeBSD system on my LAN that then uploaded the results to my web site. I figured I would give it a shot on my Mac since it should run nicely on it.

StatCVS output on my web site

Setup was truly painless. Java was already installed, so I just had to download the single jar file from the StatCVS site and unzip it into the directory I created for this purpose. I changed the paths in the shell scripts I used on FreeBSD to match the paths on OS X, added them to my crontab (same process as in FreeBSD), and it's running great.

Blows away my Pentium 4 1.6 GHz FreeBSD box that was running it. It took from 45 seconds to upwards of a minute depending on the size of the repository. My 1.42 GHz G4 takes 10-15 seconds tops. Of course you can't compare clock speed between the two, and a P4 1.6 is a *much* cheaper system than the Mac mini, but I was impressed regardless because I wasn't expecting that much of a performance increase.

Example of the scripts I use to generate StatCVS:

#!/bin/bash
# update from CVS
cd /Users/cmb/statcvs/cvsroot/m0n0wall && /usr/bin/cvs -d \ :pserver:anonymous@m0n0.ch:/m0n0wall co doc
# generate CVS logs
cd /Users/cmb/statcvs/cvsroot/m0n0wall/doc && /usr/bin/cvs log > cvs-log
# generate StatCVS output
cd /Users/cmb/statcvs/output/m0n0wall && java -jar \ /Users/cmb/statcvs/statcvs.jar \ /Users/cmb/statcvs/cvsroot/m0n0wall/doc/cvs-log \ /Users/cmb/statcvs/cvsroot/m0n0wall/doc/
# upload to web site
scp -r /Users/cmb/statcvs/output/m0n0wall/ \ user@server:/usr/local/www/statcvs/m0n0wall/

I use SSH DSA keys so passwords don't need to be entered (otherwise it wouldn't run from cron).

Mac mini

2005-04-07T22:48:00.000-04:00

I got the 1.42 GHz model with a 80 GB hard drive, and 512 MB RAM. Also added the AirPort wireless card. Got the Apple keyboard and mouse as well. Everything else is stock. Total cost before tax and shipping, $810 USD.

To address the multiple "so *you're* the idiot that buys Apple's overpriced upgrades!" comments I've already gotten to date, I compared RAM prices for the upgrade from Apple and buying the RAM elsewhere. The upgrade was $70 something. The best I could find elsewhere at the time was no more than $10 USD less including shipping. For $10, I'll get my computer ready to go out of the box, thank you very much. :)

512 MB is definitely a requirement if you want to do much with the machine without slowing it to a crawl with paging to disk. (which I assumed from what I've read, and found it to be true) I've typically been running at about 300 MB RAM in use, with 5-7 applications open on average.

I ordered it on February 26, and received it on March 19. That was three days before the original estimated ship date, but three weeks is still quite a while for a system. It shipped from China, but still got here in two business days.

It's a very slick little system. I'm very pleased with it thus far.

Desktop choices/OS X

2005-04-04T01:21:00.000-04:00

One of the main things that helped finally convinced me to put up a blog is my new Mac mini. I've toyed with OS X in the past and was impressed with the GUI at least, and the BSD underneath piqued my interests further. Seemed like a good choice for a BSD admin for a desktop platform.

My main desktop and laptop are Windows XP Pro. I admin Windows networks and a couple dozen FreeBSD systems. I've tried FreeBSD and Linux as desktops. I even had my wife on Xandros for several months (amongst some other distros I tried). Xandros came closest to being a feasible platform, but still didn't stack up for neither my wife nor myself. Too many things it couldn't do, or were much less efficient.

For Windows administration, the Windows tools are the only reasonable way to go. Yeah I could remote desktop or use Citrix from BSD or Linux, but that's not a practical solution for me. I've tried it, and it makes me less produtive. Even the tools I use to admin and work with my FreeBSD servers are much better on Windows than they are on BSD/Linux! WinSCP, WinCVS, PuTTY and related keying tools (yes there's PuTTY on BSD/Linux, but it doesn't even come close), etc. All are much easier to deal with on Windows, IMO.

I have no plans of switching to OS X as my primary desktop for the same reasons. The lack of VMware on Mac is a huge detriment to it, and Virtual PC doesn't compare. I also couldn't afford the type of box I'd need as a primary desktop. My XP desktop is a dual Xeon 2.66 GHz with 2 GB RAM because I use the hell out of VMware. It's not uncommon for me to be running 8-10 simultaneous virtual machines and using 1.6-1.8 GB RAM. With the new teams feature in VMware 5, that's become even more common. I'd end up dropping over $3K USD to get a similar spec'ed Mac.

But with all that said, I am going to post some things on my use of OS X, from the perspective of a BSD user just starting on the Mac. I'm writing this now on my Mac mini. I'm using it as a desktop at my second work station at home, on the desk with the KVM for my wire rack of machines. I typically don't work at the console of these machines, but when I do, it's nice to have a desktop close at hand.

Getting started

2005-04-03T20:08:00.000-04:00

Welcome to my blog! I've thought about starting my own blog for quite a while now, and finally decided to go for it.

Why? That's what I kept asking myself, since there are a million blogs out there and several with similar content as what I'll post. I frequently find myself doing neat things with FreeBSD or other open source tools and would like to document these things. A blog is probably the easiest way to do this. Typically I'm not going to rehash things you'll see in many other places.

What type content will you see here? Well, things that interest me. I'm a Windows and FreeBSD sysadmin, network admin (mostly Cisco), and security admin and auditor. The content here will revolve around those topics for the most part.

Welcome, and look for more content to come.

Regards,

Chris Buechler
http://chrisbuechler.com