Getting ImageMagick to work in XAMPP on OS X Mavericks (10.9)

I recently upgraded my aging mid-2009 Macbook Pro from OS X 10.6.8 (Snow Leaopard) to 10.9 (Mavericks) and am reaping the usual problems of apps not working the same any more.  The latest debacle is that ImageMagick no longer works on my local web sites.

When I first installed ImageMagick on Snow Leopard, this post at Unreal Expectations was super helpful.  Unfortunately, things didn't go so well this time around.  This post is just a summary of the bumps I encountered along the way.

First of all, one thing to note is that upgrading XAMPP by running the installer for a newer version on top of an already-installed version is unreliable in my experience.  I made the mistake of not backing up the whole /Application/XAMPP folder, and ended up losing all my database tables.  Woops!  What I should have done is run a mysqldump on my databases to export all my data into sql files that I could then later import, something like so: mysqldump -u root -p a_database_name > the_data.sql .  Alas, I did not, and I have paid for it dearly.

Anyway, so I got to the point of deciding to just uninstall XAMPP and start from scratch.  I got the XAMPP 1.8.3 installer from the official website, and installed it.

First problem: re-activating my virtual hosts.

One thing I did do was save my httpd-vhosts.conf file, and all the symbolic links I'd used in the htdocs folder, so I moved those back into their respective locations in etc/extra and htdocs.

Having done so, I tried loading one of my local sites, and all it did was redirect me to the XAMPP control panel.

I'd forgotten to restart apache.  *facepalm*  So I restarted apache using the XAMPP OS X Manager app.

Still nothing.

I'd forgotten to uncomment this line in /Applications/XAMPP/etc/httpd.conf *facedesk*:
Include etc/extra/httpd-vhosts.conf

Problem #2: 403 Error

I was all excited to see my site upon a browser refresh, but I was greeted with a 403:

Access forbidden!
You don't have permission to access the requested object. It is either read-protected or not readable by the server.
If you think this is a server error, please contact the webmaster.
Error 403

There were two things going on here.  First, I hadn't reset my folder permissions in htdocs to make them read/executable by the server process.

To fix this:

sudo chgrp -R www /Applications/XAMPP/htdocs

sudo chmod -R 775 /Applications/XAMPP/htdocs

I also needed to add a Require all granted directive to my httpd-vhosts.conf file.  One note here: in my previous installation I didn't need this directive because I had included it in my httpd.conf file like so:

<Directory />

    AllowOverride none

    Require all granted

</Directory>

But that opens up access to the root server directory to anyone, which doesn't seem good.  It's not so bad because I never have this server listening on a public port, but still, I should learn good practices, and the right way to do this is to use the directive in the vhost config itself, like so:

(In httpd-vhosts.conf)

<VirtualHost *:80>

    ... other vhost stuff ...

    <Directory "/Applications/XAMPP/htdocs/xampp">

        Options FollowSymLinks

        AllowOverride All

        Require all granted

    </Directory>

</VirtualHost>

So after restarting apache again, the site loads!  Huzzah!

Progress: Re-installing ImageMagick

There are two parts to installing ImageMagick for use in apache.  First, you install the ImageMagick application.  I used MacPorts to do so:

sudo port install ImageMagick

Then you need to install the imagick extension.  I used pecl:

sudo pecl install imagick

Pecl should stick a file called imagick.so into /Applications/XAMPP/xampfiles/lib/php/extensions/no-debug-non-zts-<somenumber>/.  You may need to change the file permissions of this file if it's not executable already:

sudo chmod 755 /Applications/XAMPP/xampfiles/lib/php/extensions/no-debug-non-zts-20100525/imagick.so

And make sure there is a line like this in /Applications/XAMPP/etc/php.ini:

extension="imagick.so"

And while you're in php.ini, make sure that extension_dir is set to the path to the folder in which imagick.so is now sitting.

Problem the next: imagick.so doesn't load

After restarting apache and attempting to load a page that uses ImageMagick, I got the classic error telling me that ImageMagick still isn't working:

Fatal error: Class 'Imagick' not found in blah blah blah

I checked my phpinfo (by using the php -i command) and noticed that indeed there is no reference to imagick anywhere.  Digging a bit deeper, I checked my php error log (the location of which is determined by the error_log setting in the php info) and noticed the following error:

PHP Warning:  PHP Startup: Unable to load dynamic library '/Applications/XAMPP/xamppfiles/lib/php/extensions/no-debug-non-zts-20100525/imagick.so' - dlopen(/Applications/XAMPP/xamppfiles/lib/php/extensions/no-debug-non-zts-20100525/imagick.so, 9): Library not loaded: /opt/local/lib/libfreetype.6.dylib

  Referenced from: /opt/local/lib/libMagickWand-6.Q16.1.dylib

  Reason: Incompatible library version: libMagickWand-6.Q16.1.dylib requires version 17.0.0 or later, but libfreetype.6.dylib provides version 15.0.0 in Unknown on line 0

The problem here is that the ImageMagick library I'm using links a different version of libfreetype.6.dylib than what is used in my XAMPP installation.  The fix is to copy the Macports version of this library into the XAMPP directory (but make a backup of the XAMPP one before you do this, of course):

cp /opt/local/lib/libfreetype.6.dylib /Applications/XAMPP/xampfiles/lib

Problems, more problems: apache no longer restarts

When I went to try restarting apache this time, it didn't even stop for me any more.  So I went to the commandline:

apachectl start

Yielded a similar error as before.  This time it's complaining about libpng:

Syntax error on line 522 of /Applications/XAMPP/xamppfiles/etc/httpd.conf: Syntax error on line 10 of /Applications/XAMPP/xamppfiles/etc/extra/httpd-xampp.conf: Cannot load modules/libphp5.so into server: dlopen(/Applications/XAMPP/xamppfiles/modules/libphp5.so, 10): Library not loaded: /opt/local/lib/libpng15.15.dylib\n  Referenced from: /Applications/XAMPP/xamppfiles/lib/libfreetype.6.dylib\n  Reason: Incompatible library version: libfreetype.6.dylib requires version 33.0.0 or later, but libpng15.15.dylib provides version 25.0.0

So I do the same thing as previously, backing up the libpng in the XAMPP folder and moving in the Macports one:

cp /opt/local/lib/libpng15.15.dylib /Applications/XAMPP/xampfiles/lib

Now apache starts and stops happily.  But there's another php error now again.  This time we need to update libexpat:

cp /opt/local/lib/libexpat.1.dylib /Applications/XAMPP/xampfiles/lib

And at last, I have a functioning apache and a functioning ImageMagick extension in OS X Mavericks!  That seemed like too much work.

Disappearing Advanced Custom Fields

So at work today I received an email from a client who had just run the WordPress upgrade from 3.6.0 to 3.6.1 for a site we support.  Some fields managed by the Advanced Custom Fields plugin had disappeared after the upgrade.  Great...

I dug around for a bit in the admin site and discovered that most of the ACF fields remained intact, but 3 Text fields and one Yes/No field was no longer showing up from one particular Field Group we had set up.

So I took a look at the database, and here is what I learned:

ACF Tables

ACF is written in a way that allows it to work without creating additional tables in your WordPress instance.  This presumably gives it the advantage of sneaking easily through WordPress upgrades (although the topic at hand makes me wonder about this...), but it makes for some wonky use of the standard WordPress tables.

ACF stores all its data in the wp_postmeta table, except for Field Groups, which are in the wp_posts table with post_type='acf'.  Fields are associated with their Field Groups via the post_id of the main Field entry in wp_postmeta

ACF Records in wp_postmeta

Each ACF Field has one main record in the wp_postmeta table, plus two records for each use of that Field in a post or page or whereever you set the ACF Field to be used.  Let's take a look at the records associated with one of the ACF Fields that still existed in the WordPress instance I was dealing with, the Description field:

In wp_postmeta there is one record with the following data:

meta_id: 41
post_id: 15
meta_key: field_51e859f459a64
meta_value: a:12:{s:3:"key";s:19:"field_51e859f459a64";s:5:"label";s:11:"Description";s:4:"name";s:11:"description";s:4:"type";s:8:"textarea";s:12:"instructions";s:37:"Enter the description of this product";s:8:"required";s:1:"1";s:13:"default_value";s:0:"";s:11:"placeholder";s:0:"";s:9:"maxlength";s:0:"";s:10:"formatting";s:2:"br";s:17:"conditional_logic";a:3:{s:6:"status";s:1:"0";s:5:"rules";a:1:{i:0;a:3:{s:5:"field";s:19:"field_51e86279a9bee";s:8:"operator";s:2:"==";s:5:"value";s:1:"1";}}s:8:"allorany";s:3:"all";}s:8:"order_no";i:1;}

The meta_id is just the unique identifier for this record in the wp_postmeta table.
The post_id is the id of the Field Group post in the wp_posts table to which this Field is associated.
The meta_key is a unique identifier generated by ACF for this field.
Finally, that scary looking meta_value is just a serialized PHP array that constitutes the settings for the ACF Field.  (If you look closely, you can see that this particular field has the label 'Description' and is a 'textarea' field.  This field happens to be conditional on another field, which manifests down in the "rules" sub-array.  You can see the field id which this field is dependent on, and that it's dependent on the other field having a value of 1, and so on.  The point of this post isn't to go too far into the bowels of ACF meta data, though.  Just enough to fix the problem that I had.  So let's move on.)

There are also a bunch of records that look like this in the database (I'll leave out meta_id here since we already know what that means):

post_id: 14
meta_key: description
meta_value: This is a description

and this:

post_id: 14
meta_key: _description
meta_value: field_51e859f459a6

There is one pair like this for each use of the field on the web site.  Here, the post_id is not the id of the Field Group any more, but rather the id of the actual post in which this ACF Field is being used (this could be a page id or taxonomy_term id too, depending on the ACF Field's settings).  The meta_key in the first case is just the 'name' you see in the meta_value of the main ACF record above (a la s:4:"name";s:11:"description";).  The meta_value in the first case is the value of this instance of the field.

Now why that second record?  Well, there needs to be a way to associate this instance of a description field with the main description field record, and the second record here serves that purpose.  I'm guessing that the '_' is just a convention used by the ACF author to make it easy to determine which record is contains the actual value and which one contains the association info.

Armed with that info, we're ready to tackle the problem I encountered.

The Problem

When I looked at the wp_postmeta records associated with one of my missing fields (let's use the 'brochure_url' field as an example), I noticed that while there were indeed still lots of 'brochure_url' and '_brochure_url' records, there was no longer a 'field_xxxxxxxxxxxxx' record for this field.  Uh oh!

I have no idea why that record went missing.  Perhaps the upgrade script tried to do some database cleaning that doesn't mix well with ACF, or perhaps my client (or even myself?) accidentally deleted the field in the admin interface.  Either way, for each of my missing fields, the main ACF Field record was gone but all the data pair records were still there.

The Fix

The fix turned out to be fairly easy, with one little trick requiring some work directly in the database.  All I did was go back to the WordPress admin and recreate the missing fields, being careful to use exactly the same name as before.  (I could easily determine the name since the data pair records were still in the database.)

After recreating the fields, they all have unique identifiers that no longer matched the ones that were used by these fields before.  For example, my '_brochure_url' records all had the meta_value 'field_51e859f459a64', but my new main record for brochure_url has the meta_key 'field_51e860fa1ed53'.

Now for the trick:  I went into my database (was stuck with phpMyAdmin), found the main field_51e860fa1ed53 record in wp_postmeta, and changed two bits of data to 'field_51e859f459a64':  the meta_key, and the corresponding portion in the serialized meta_value for this record.

Et voila!  The site was back to normal.

So just to be clear, here's the 'before' picture:

The association record:

meta_key: _description 
meta_value: field_51e859f459a64

The main field record:

meta_id: 41
post_id: 15
meta_key: field_51e860fa1ed53
meta_value: a:14:{s:3:"key";s:19:"field_51e860fa1ed53";s:5:"label";s:12:"Brochure URL";s:4:"name";s:12:"brochure_url";s:4:"type";s:4:"text";s:12:"instructions";s:35:"A URL to a brochure pdf or web site";s:8:"required";s:1:"0";s:13:"default_value";s:0:"";s:11:"placeholder";s:0:"";s:7:"prepend";s:0:"";s:6:"append";s:0:"";s:10:"formatting";s:4:"none";s:9:"maxlength";s:0:"";s:17:"conditional_logic";a:3:{s:6:"status";s:1:"0";s:5:"rules";a:1:{i:0;a:3:{s:5:"field";s:4:"null";s:8:"operator";s:2:"==";s:5:"value";s:0:"";}}s:8:"allorany";s:3:"all";}s:8:"order_no";i:5;}

And here's after:

The association record:

post_id: 14
meta_key: _description
meta_value: field_51e859f459a64

The main field record:

meta_id: 41
post_id: 15
meta_key: field_51e859f459a64
meta_value: a:14:{s:3:"key";s:19:"field_51e859f459a64";s:5:"label";s:12:"Brochure URL";s:4:"name";s:12:"brochure_url";s:4:"type";s:4:"text";s:12:"instructions";s:35:"A URL to a brochure pdf or web site";s:8:"required";s:1:"0";s:13:"default_value";s:0:"";s:11:"placeholder";s:0:"";s:7:"prepend";s:0:"";s:6:"append";s:0:"";s:10:"formatting";s:4:"none";s:9:"maxlength";s:0:"";s:17:"conditional_logic";a:3:{s:6:"status";s:1:"0";s:5:"rules";a:1:{i:0;a:3:{s:5:"field";s:4:"null";s:8:"operator";s:2:"==";s:5:"value";s:0:"";}}s:8:"allorany";s:3:"all";}s:8:"order_no";i:5;}

Note the two places that I had to change in the main ACF Field record so that the field's unique id corresponded to the old unique id.