Category Archives: Mac OS X

Unlocking a protected PDF on Mac OS X

Recently I needed to demonstrate proof of purchasing something via my credit card statement. Easy enough, I download my most recent statement as a PDF file from American Express. Then I wanted to use Adobe Acrobat Pro’s nifty redaction features to redact all the irrelevant information from the appropriate page of the bill. Except Amex has decided that the statement should be a protected PDF, which means you can view it but cannot change it. This is of course totally bogus DRM, it’s my statement afterall! I suppose they hope to curb statement forgeries, but as anyone akamai knows: if I can view it, I can edit it. I think on Mac OS X used to ignore DRM and let you edit protected PDFs, but doesn’t seem to on Snow Leopard.

I hunted around for a tool to unlock the PDF. There are lots of tools for Windows, which didn’t interest me. One person suggested opening the PDF and “printing” it to a PDF, but Adobe has disabled those features of the Print dialog box on Mac OS X (presumably since it would allow trivial circumvention of the DRM).

PDFKey Pro looks like a reasonable option for Mac OS X, but it is $25 which seems kinda steep for a single use. They have a downloadable demo, but it will just create an unlocked version of the first page of the PDF, which wasn’t the page I wanted. And of course I can’t edit the source PDF because it is protected, so the demo wasn’t useful to me.

Then I came upon MuPDF, which is a “lightweight PDF viewer and toolkit written in portable C”. It has an X11 GUI component, as well as command line tools. One of the command line tools is “pdfclean”, which will remove the DRM from a PDF.

Unfortunately, MuPDF isn’t in MacPorts yet, so I had to compile it by hand. It uses the Perforce jam tool instead of make, and has three library dependencies: zlib, libjpeg, and freetype2. Luckily, all of these are available in MacPorts, so I was able to install them and then edit the Jamrules file to point at the MacPorts location. Here is the updated section of Jamrules:

if $(OS) = MACOSX
    Echo Building for MACOSX ;

    BUILD_X11APP = true ;

    CCFLAGS = -Wall -std=gnu99 -I/opt/local/include -I/opt/local/include/freetype2 ;
    LINKFLAGS = -L/usr/X11R6/lib -L/opt/local/lib ;
    LINKLIBS = -lfreetype -ljpeg -lz -lm ;
    APPLINKLIBS = -lX11 -lXext ;

    if $(BUILD) = debug   { OPTIM = -g -O0 -fno-inline ; }
    if $(BUILD) = release { OPTIM = -O3 ; }

    if $(HAVE_JBIG2DEC) { LINKLIBS += -ljbig2dec ; }
    if $(HAVE_OPENJPEG)    { LINKLIBS += -lopenjpeg ; }

pdfclean worked like a charm, removing the DRM from the statement. After that I was able to redact the statement without incident.

Perhaps in my copious spare time I will make a MuPDF portfile for MacPorts, but until then perhaps this will help others who want an open source way to remove bogus PDF DRM.

Snow Leopard Server upgrade hoses Apache config

I recently upgraded an Xserve running Mac OS X Server from 10.5 (Leopard) to 10.6 (Snow Leopard). The upgrade was mostly uneventful, except for the web server configuration. We have a lot of manual changes (two instances of Plone using mod_rewrite, some local directories that are served up as is, and user directories). This is a rundown on the changes I had to make after the upgrade was complete to restore web service.

As part of the upgrade, Snow Leopard Server runs a script that reads the Apache config files and tries to bring them into compliance with the default configuration under Snow Leopard. The result of all the configuration munging (for all services) can be found in /Library/Logs/Setup.log. I found several problems with the changed Apache configuration. Kindly, the configuration munging script leaves unmodified copies of all the files it modifies with the suffix “.saved-Pre-SnowLeopard-unmodified”, so it is straightforward to diff the old configuration with the new one to see what changed.

First, mod_userdir was commented out in httpd.conf, preventing “~username” URL rewriting. Snow Leopard also adds an “mod_apple_userdir” (which supposedly improves security of this feature [PDF]), but also left that disabled. So by default, “~” URLs just stop working, with no notification beyond a bullet point in one of the supplemental PDFs. This is poor form, IMHO. I ended up re-enabling mod_userdir.

The next problem was the configuration file munger decided to remove all RewriteRule lines from each of the virtual host configurations. Obviously, this broke many things, but it was easy enough to copy each line back from the original configuration.

Even after fixing those issues, web service wasn’t working properly. I noticed that there was a “0000_any_80_.conf” configuration in the “sites” subdirectory (in addition to “0002_any_80_.conf”), which wasn’t being used in Leopard, but appeared active in Snow Leopard. I moved this config to the “sites_disabled” directory, and that made things much better. I’m not sure if this was some vestige of an upgrade from Tiger to Leopard, but it was causing problems.

Finally, the config munger added “ServerAlias *” to my first virtual host configuration (prefixed with “0000”). While this might be appropriate if this was the default host, as a virtual host it grabbed all URLs and prevented them from matching virtual hosts with later sequence numbers. Removing this line fixed the last issue (which was that all requests went to the first virtual host).

Admittedly, Apple faces a difficult proposition in performing automated server OS upgrades. Many of the OS services are based on industry standard open source packages that overwhelmingly use custom text configuration files. Advanced administrators will frequently want to edit those files by hand to install  3rd party open source packages that build on those services. But Apple also provides GUI tools to make things easier, tools that must read and write the configuration files. This leads to the desire to keep the configuration files canonicallized for easier parsing. So at the end of the day, some poor engineer has to write an upgrade script to read all these (possibly arbitrarily modified) configuration files and bring them into line with the versions that ship with the new OS version.

That said, it seems like the upgrade scripts could do a better munging job. Turning off modules is uncool, as is dropping lines from the config on the ground.

Java 1.6 & Eclipse on Mac OS X

These are some notes on using Java 1.6 on Mac OS X with Eclipse.

Mac OS X lagged behind for some time on adopting Java 1.6, but it was finally released as Java for Mac OS X 10.5 Update 1. Unfortunately, this release has some limitations: it only runs in 64-bit mode and only on Mac OS X 10.5 (Leopard). This meant no Java 1.6 support on Mac OS X 10.4, no support for PowerPC systems, and no support on the early 32-bit Intel Macs released in 2006.

To make Java 1.6 the default Java execution environment, run the Java Preferences application (found in the Utilities subfolder of the Applications folder). There you can drag the Java versions into any order you wish for both applications and applets. So to default to using Java 1.6 for everything, you can just drag Java SE 6 to the top of both the application and applet lists. The Java Preferences application will change around symlinks to correspond to your new choices. Mac OS X 10.6 (Snow Leopard) appears to include only Java SE 6, but includes both 32 and 64-bit versions, so it should allow Java 1.6 to be run on the oldest Intel Macs (no PPC because Snow Leopard does not support PPC).

Some command line scripts expect the JAVA_HOME environment variable to be set to the directory that contains the Java distribution being used. Also, some scripts might require different versions of Java. To support this, Apple introduced a new utility called /usr/libexec/java_home. By default, java_home just returns the home value appropriate based on the selection in the Java Preferences utility, but command line arguments can request different versions (see the manual page linked earlier). The output of java_home is intended to be assigned to the JAVA_HOME environment variable in a shell initialization script (like .profile or .cshrc).

This brings us to Eclipse, the Java development environment. Eclipse can be used to develop for multiple Java versions. When creating a Java project, Eclipse prompts for the desired JRE version in the new project dialog box. To ensure that Eclipse tracks the language differences between versions, also select the desired version in Preferences->Java->Compiler->Compiler compliance level to match your project.

However, despite these changes, Eclipse itself does not necessarily run using the Java version being used for the project being developed. Initially, the latest version of Eclipse (3.5) was released only in 32-bit mode for both Carbon (an older and deprecated Mac OS API) and Cocoa (the modern Mac OS API). Since Leopard only supported Java 1.6 in 64-bit mode, this meant that Eclipse was always running under Java 1.5. With the release of Eclipse 3.5.1, there are now 64-bit Cocoa downloads available, and these will run under Java 1.6. For some users, this may not be important, but for those doing Eclipse tool development (such as the Hackystat Eclipse sensor) it is very helpful. I use JAXB in some of my Ant build scripts, and JAXB is built into Java 1.6 but not Java 1.5. When running Eclipse in Java 1.5, many of my Ant scripts report spurious errors about being unable to locate JAXBException, but these all vanish when running Eclipse under Java 1.6. I should note that at least some people disagree with this advice, and suggest using the 32-bit Cocoa version on Leopard instead of the 64-bit version. Everyone apparently agrees that on Snow Leopard you want the 64-bit Cocoa version of Eclipse (unless you are on a 32-bit Intel Mac).

So if you want to use Java 1.6 for your application and run Eclipse under Java 1.6 there are two options:

  • Use Mac OS X 10.5 Leopard (with all software updates) on a Core 2 Duo Intel Mac, with Java Preferences set to use Java SE 6 as the default Java version, and use the 64-bit Cocoa version of Eclipse 3.5.1.
  • Use Mac OS X 10.6 Snow Leopard on an Intel Mac, and use the 64-bit Cocoa version of Eclipse 3.5.1 [note, I have not specifically tested this configuration, but it should work. Confirmed, this works]

ssh-agent and Mac OS X 10.5 Leopard

I’ve been using ssh-agent since I started using Mac OS X (10.1), it being the best combination of security and convenience when using ssh and scp/sftp. During that time I have been using some modifications to my .tcshrc file to start ssh-agent when I log in and set up the ssh environment variables properly.

However, Mac OS X 10.5 aka Leopard now has nice ssh-agent support built-in that my startup file was overriding. I wanted to understand more about the new ssh-agent support in Leopard before I went removing my ssh-agent code. Sure enough, there are a variety of blog posts that explain the situation. This post at Ormset i Noreg explains the situation, and discusses complications if you have been using the open source SSHKeyChain GUI application to manage your ssh-agent under Tiger.

Dave Dribin goes into more depth on exactly what is going on with the keychain. He also has a followup post on how to keep your passphrase stored in a separate keychain to make it less vulnerable that I plan to follow. He also has written some code to support one of SSHKeyChain’s features: removing keys from the agent when the computer is put to sleep.

It would be nice if SSHKeychain was rewritten to be more of a support service for the Leopard SSH stuff, but based on some discussion on the mailing list it appears that the author has been looking for a new maintainer for a few months.

Week 8: BibTeX & BibDesk for annotated bibliographies

I made substantially less progress in the past week than expected for two reasons. First, a crisis came up in LILT that required me to urgently work on SocialSense. Second, as part of the Sustainable Saunders Energy Group, I was notifying the occupants of the sixth floor of the Saunders building about upcoming nighttime air conditioning shutdown that should save at least $100K annually. Hopefully both situations have subsided for now and I can focus on the sustainability research this week.

I was looking for a way to get my literature review notes into my proposal document (per Philip’s request) in a sane and maintainable way. As I read each paper, I make notes (usually just bullet points) and once I have reached a stopping point (like the end of a semester) I assemble them into something conceptually coherent. I decided the best way to do this is to enter all the papers I read into my proposal BibTeX database, and place my notes in the annote field. BibDesk is a sweet Mac OS X GUI for maintaining BibTeX databases, and has easy access to the annote field. Then the key was getting the notes into the document. There are BibTeX styles (.bst files) that provide different bibliography styles, and some will print the annote fields of each referenced item, producing an annotated bibliography. I ended up using the annotation style, which ships as a standard part of TeX Live 2008. There are other annotated bibliography styles that could be installed, but I decided to go with the built-in one for now. Obviously I will switch back to the normal bibliography style when the literature review section is done.

Planned items from last week:

  • Read 2 papers from literature review list
    • Understanding mobility based on GPS data
      • Not done
    • Learning Transportation Mode from Raw GPS Data for Geographic Applications on the Web
      • Not done
  • Start building table of possible sensor inputs
    • Not done
  • Add publications from PET workshop paper to BibTeX
    • Not done

Other accomplishments this week:

  • Fixed formatting on my research portfolio per Philip’s suggestions
  • Moved literature review notes into proposal document per Philip’s request via annotated BibTeX entries

Hours worked: 6 (target: 15 hr)

Plans for coming week:

  • Read 4 papers from literature review list
    • Understanding mobility based on GPS data
    • Learning Transportation Mode from Raw GPS Data for Geographic Applications on the Web
  • Start building table of possible sensor inputs
  • Add publications from PET workshop paper to BibTeX

Pointers to work products:

Cool links:

  • Caffeine, a Mac OS X application that allows disabling sleep, screen dimming, and the screen saver via a menu bar item. This is really handy for presentations, so you don’t have to keep moving the mouse to keep your system awake. There’s even a timer functionality so it only stays “awake” for a fixed amount of time, so you don’t have to worry about forgetting to turn it off. Freeware.

TeX/LaTeX on Mac OS X

I just installed TeX on my MacBook Pro, and boy is it a lot easier than installing it on a server a decade ago. Just download MacTeX-2008 from TUG and you’re basically done. MacTeX installs a nifty little System Preferences pane that allows you to have different versions or distributions of TeX installed and switch between them with a mouse-click. It even sets your PATH and MANPATH variables automagically using the /etc/paths.d directory facility in Leopard. In my case that didn’t work because I have a custom .tcshrc, but the “What Is Installed” document in /Applications/TeX/Utilities/Documents explained exactly what they are doing so it was easy to update my path.

It’s good to be using TeX again, though I’m still learning the new stuff. I’m doing latex, latex, bibtex, latex, dvipdf, but I think the new way would be to use pdflatex. However, it seems to choke on the EPS file I have as part of the UH thesis example document.

MarcoPolo, context-aware location switching for OS X

MarcoPolo is a very cool little GPL-licensed application that allows you to define different contexts (which are often locations) and switch between them based on fuzzy rule matching. The rules are based on evidence like what WiFi access points are nearby, what monitor your are connected to, what USB devices are attached, etc. When switching to a context, MarcoPolo can perform a variety of actions, like changing the default printer, changing your iChat status message, etc.

So the most common usage pattern is to create contexts for the different locations you use your computer, use the rules to detect your location, and then set up actions you want to take when you arrive or leave a location. Very cool stuff, and it’s open source too boot!

VMware Fusion, Apple Boot Camp, and Windows Activation

I’m trying to get Windows XP SP2 running on my MBP using both Boot Camp (to be able to run Windows natively) and virtualized using VMware Fusion, but sharing the same Windows installation. This turns out to be complicated by Windows annoying product activation system. When run under Fusion, Windows sees different hardware than when running natively on Boot Camp, so it wants you to reactivate.

According to this thread in the VMware forums and page 8 of the Fusion Getting Started guide (PDF), the solution is to activate things in a particular order:

  1. Install Windows XP SP2 using Boot Camp
  2. Activate Windows under Boot Camp
  3. Install Fusion, picking the Boot Camp as the virtual machine
  4. Install the VMware Tools
  5. Reboot the virtual machine
  6. Activate Windows under Fusion

Now one should be able to run under Boot Camp or Fusion with no reactivation problems. I am about to try the Fusion side now.

Note for Vista users, Fusion 1.0 isn’t able to finesse the Windows activation issue (this is mentioned in the release notes).

Update: When I tried to activate in Fusion, Windows told me my license key had used up all its reactivations. So, I was forced to use their automated telephone system to activate, which was comically bureaucratic to Brazil proportions. You have to repeat 9 groups of 6 numbers [sic!!] to an automated attendant, which then tells you 6 groups of 6 numbers that you have to type in. It boggles my mind that Windows users put up with this kind of crap. Anyway, that finally worked, but then when I booted back into Windows via Boot Camp it needed reactivation. Luckily, I could do it via the Internet activation option this time, and now Windows seems happy in both Boot Camp and Fusion. bleargh.

Mac OS X hostname determination

I recently came across this information in an Apple PDF worksheet and couldn’t find anything via Google that really laid out the full situation, so I thought I would write this up for posterity.

In Mac OS X 10.4.x Server (and the client version I believe [and possibly earlier versions], by default your hostname is determined dynamically. This is controlled by the following line in the file /etc/hostconfig:


(note that in my Mac OS X 10.4.10 client this line is absent from /etc/hostconfig so I assume that absence defaults to automatic as well). If you want to your hostname to be static, just change -AUTOMATIC- to whatever fully qualified hostname you want like

When the hostname is determined automatically, the OS steps through this list of possibilities and goes with the first valid name found:

  1. The name provided by the DHCP or BootP server for the primary IP address
  2. The first name returned by a reverse DNS (address-to-name) query for the primary IP address
  3. The local hostname (set in the Sharing pane of System Preferences)
  4. The name localhost

So this means if you get different IP addresses at home, work, and on the road you will see your hostname bounce around, which can be disconcerting if you aren’t expecting your shell prompt to change (wave to Philip :)).

Update: I changed the hostname of a Mac OS X 10.5 Server, but in certain places (such as the output of the hostname command) I would still get the old hostname. Turns out that there is a preference file that didn’t get updated for some reason:/Library/Preferences/SystemConfiguration/preferences.plist. I found a reference to the old hostname there, and after editing the file and rebooting, it appears that the last traces of the old hostname are gone.

Thanks to this post from this thread on for the pointer on how to fix this.

Installing MacPorts from the Command Line

I wanted to install MacPorts on an Xserve using ssh. This involved doing two things I had never done from the shell: mounting a disk image and running an installer. Here are some quick instructions in case someone else needs to do this. It is assumed that you have already installed Apple’s X11 and the X11SDK per the MacPort install instructions.

  • ssh to the Mac OS X system
  • Download MacPorts
  • Mount disk image
    • hdiutil attach MacPorts-1.5.0-10.4.dmg
  • Run the installer
    • sudo installer -verbose -pkg /Volumes/MacPorts-1.5.0/MacPorts-1.5.0.pkg -target /
  • Add /opt/local/bin (and probably /opt/local/sbin) to your path in whatever way your shell requires
  • Update MacPorts to the latest version
    • sudo port -v selfupdate
  • Unmount the disk image
    • hdiutil detach -verbose /dev/disk4
    • The device was displayed when you mounted the image, if you forgot it then run mount