WordPress Performance Tips for Big Sites

One of media websites which we administer has 14,000 posts and close to 400,000 comments. Is WordPress any good as running a big site like this? Does it scale well?

Read some of our tips how to optimize WordPress for large sites below:

Post revisions

This is a weakness of WordPress. On this site there are over 30,000 revisions for the 14,000 posts. That makes the table bigger and it's slower to search in it. WordPress users realized this three years ago.

Tip #1

We backed up the the wp_posts table and then used a simple MySQL command to remove old post revisions. This decreased the table size from 400MB to 120MB:

DELETE FROM `wp_posts` WHERE post_type = 'revision' AND post_date NOT LIKE '2012-%'

Long-term solution

There are WordPress plugins which can limit the number of revisions per post. We think that the WordPress code should be improved and the revisions should be stored in a different table to maximize speed. You can support this on WordPress bug tracking.

Large number of comments

We need to be extra careful specially with the wp_comments table which has close to 400,000 comments and 320MB. We often find speed issues with MySQL queries which take only a couple of milliseconds on smaller sites.

WordPress 3.2 added a count of comments into the WordPress Admin Bar which shows up for logged in Administrators and Editors when browsing the site.

Comment count in WordPress Admin Bar

Counting the comments actually takes a lot of time on our big database. Here's the query and it's duration in log:

SQL query for counting of the comments

SELECT comment_approved, COUNT(*) AS num_comments FROM wp_comments GROUP BY comment_approved;

In the above case, it takes 0.3 seconds, while all the other queries are done in 0.05 - 0.001 seconds.

Tip #2

To do this kind of audits use WPDB Profiling plugin for WordPress. It shows you all sorts of information in the site footer and you can turn it on and off as you like.

Beware: this plugin turns off post revisions and autosaves when activated, you need to turn it off once you are finished or change it's settings.

It's clearly the slowest query when the site is loading. And this site has up to 20 editors who like to come to the site and read the comments to their articles, so it affects the performance. Keep in mind that this information is shown on multiple places in WordPress Admin Interface.

We created our own queries to do this count. It's 5 queries instead of 1, but they are faster. Just try to test them:

SELECT COUNT(comment_ID) FROM wp_comments WHERE comment_approved = 'trash'
SELECT COUNT(comment_ID) FROM wp_comments WHERE comment_approved = 'spam'
SELECT COUNT(comment_ID) FROM wp_comments WHERE comment_approved = '0'
SELECT COUNT(comment_ID) FROM wp_comments WHERE comment_approved = 'post-trash'
SELECT COUNT(comment_ID) FROM wp_comments

The last query counts all the commands, so we subsctract the previous counts. Here are our results - 0.042144 seconds:

SQL query for counting of the comments improved

That's a big improvement over 0.3 seconds duration with the standard query.

Tip #3

You can also download a code and test it on your site, just put it into functions.php file of template here.

If you want to test above queries, replace SELECT with SELECT SQL_NO_CACHE to make sure no MySQL caching will be used.

We also opened a bug tracking ticket for this on WordPress Trac - Speeding up Dashboard and Comment moderation SQL load.

Finding slow database queries with MySQL tools

Another way of finding the slow queries is using this MySQL command. It's best used when you have SSH access to the site.

1. Watch the server load using top.
2. When you see that mysql process is taking too much of the CPU, just try to list running queries with following command in MySQL console:

mysql> SHOW PROCESSLIST;

+-------+------+---------+------+--------------+------------------------------------------------------------------------------------------------------+
| Id    | User | db      | Time | State        | Info                                                                                                 |
+-------+------+---------+------+--------------+------------------------------------------------------------------------------------------------------+
| 59462 | site | site_db |    0 | NULL         | SHOW PROCESSLIST                                                                                     |
| 61208 | site | site_db |   62 | Sending data | SELECT ID FROM wp_posts AS a LEFT JOIN (SELECT post_id FROM wp_postmeta WHERE meta_key='...') AS b   |
| 61228 | site | site_db |   25 | Locked       | UPDATE `wp_postmeta` SET `meta_value` = '1327484262:39' WHERE `post_id` = 66955 AND `meta_key` = '_e |
| 61238 | site | site_db |   16 | Locked       | SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (17992)                       |
| 61241 | site | site_db |   16 | Locked       | SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (12931)                       |
| 61249 | site | site_db |   11 | Locked       | SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (34465)                       |
| 61251 | site | site_db |   11 | Locked       | SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (5209)                        |
| 61257 | site | site_db |    6 | Locked       | SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (34465)                       |
| 61258 | site | site_db |    5 | Locked       | SELECT meta_id FROM wp_postmeta WHERE meta_key = '....' AND post_id = 24661                          |
| 61262 | site | site_db |    1 | Locked       | SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id IN (5367)                        |
+-------+------+---------+------+--------------+------------------------------------------------------------------------------------------------------+

In the above list, there is some nasty query which is taking 62 (!) seconds to get executed and the other queries are just waiting until it's finished. Yes, the site was in real trouble when the above list was saved.

There is also an MySQL option called "log slow queries", but we newer had a success with it.

Here's the query in it's full beauty. Since it's using subqueries, it's slow and hard to optimize:

SELECT ID FROM wp_posts AS a LEFT JOIN (SELECT post_id FROM wp_postmeta WHERE meta_key='fb') AS b ON a.ID=b.post_id WHERE b.post_id IS NULL;

We re-coded the plugin and this bad query is no longer used. This was some weird query in some old plugin which was running as WordPress Cron job, so it's not visible in WPDB Profiling which was described above.

Tip #4

If the query seems to complicated, try to move some of the processing into PHP.

Use indexes!

If you see a slow query, try to use MySQL EXPLAIN command on it. Here's the query:

SELECT count(*) FROM wp_comments AS c JOIN wp_posts AS p ON c.comment_post_ID = p.ID WHERE c.user_id = '1079' AND c.comment_approved = '1' AND p.post_status = 'publish' AND comment_content REGEXP '[[::]]' = 1

Here's the output of the EXPLAIN command. Notice the big number in the "rows" column. It means that MySQL has to examine 377,606 rows in the wp_comments table - that's all the comments on the site.

mysql> EXPLAIN SELECT count(*) FROM wp_comments AS c JOIN wp_posts AS p ON c.comment_post_ID = p.ID WHERE c.user_id = '1079' AND c.comment_approved = '1' AND p.post_status = 'publish' AND comment_content REGEXP '[:asdf:]' = 1;
+-------------+-------+--------+------------------------------------------------------------+------------------+---------+------------------------+--------+-------------+
| select_type | table | type   | possible_keys                                              | key              | key_len | ref                    | rows   | Extra       |
+-------------+-------+--------+------------------------------------------------------------+------------------+---------+------------------------+--------+-------------+
| SIMPLE      | c     | ref    | comment_approved,comment_post_ID,comment_approved_date_gmt | comment_approved | 62      | const                  | 377606 | Using where |
| SIMPLE      | p     | eq_ref | PRIMARY                                                    | PRIMARY          | 8       | site.c.comment_post_ID |    1   | Using where |
+-------------+-------+--------+------------------------------------------------------------+------------------+---------+------------------------+--------+-------------+
2 rows in set (0.00 sec)

The SQL query is using user_id in the WHERE clause. And there is no index which would contain this (see possible_keys in above output, you can also execute "SHOW INDEXES IN wp_comments;").

So we create a new index which will combine two fields from the WHERE clause:

CREATE INDEX userid_approved_index ON `wp_comments` (`user_id`,`comment_approved`)

The table index size will increase a bit, but SQL will search only 1,423 rows when executing our query, because it's able to use our new index:

mysql> EXPLAIN SELECT count(*) FROM wp_comments AS c JOIN wp_posts AS p ON c.comment_post_ID = p.ID WHERE c.user_id = '1079' AND c.comment_approved = '1' AND p.post_status = 'publish' AND comment_content REGEXP '[:asdf:]' = 1;
+-------------+-------+--------+----------------------------------------------------------------------------------+-----------------------+---------+----------------------------------+------+-------------+
| select_type | table | type   | possible_keys                                                                    | key                   | key_len | ref                              | rows | Extra       |
+-------------+-------+--------+----------------------------------------------------------------------------------+-----------------------+---------+----------------------------------+------+-------------+
| SIMPLE      | c     | ref    | comment_approved,comment_post_ID,comment_approved_date_gmt,userid_approved_index | userid_approved_index | 70      | const,const                      | 1896 | Using where |
| SIMPLE      | p     | eq_ref | PRIMARY                                                                          | PRIMARY               | 8       | mondoweiss_net.c.comment_post_ID |    1 | Using where |
+-------------+-------+--------+----------------------------------------------------------------------------------+-----------------------+---------+----------------------------------+------+-------------+

Tip #5

When you create a plugin which uses custom SQL queries, have look at the tables and make sure the fields which you used in WHERE or ORDER BY statements are indexed.

Add more indexes to the table if necessary (but then be careful if you ever reinstall WordPress).

Another example would be sorting comments by comment_date. comment_date field is not indexed, but comment_date_gmt is. So use that one and your queries will be faster.

• Check out the indexes on wp_comments table in Database Description on WordPress.org Codex.
• Read more about indexes in MySQL 5.0 Reference Manual

Results

Check out the load of our server. It dropped significantly once we added these tweaks.

Reduced load

VPN for OS X: Witopia 2012 review (from a three year user)

It used to be really simple to get a VPN for Mac. You'd just go and sign up at Witopia.net and you'd get an inexpensive and reliable VPN with nodes all over the world. Witopia would work with the built-in networking inside your Mac, specifically PPTP protocol.

Here's all the reasons you might want a VPN.

The problem was Bill Bullock was obsessive about our security/anonymity. So the customer portal didn't really work or hardly existed. Support was over email and a bit hope for the best. Your email address had a single subscription.

Worse yet, when you would go to renew your subscription, you had to create a new account and login.

From the beginning there were two services: Personal, which always included PPTP and now includes ipsec and L2TP and Pro which was twice the price and included OpenVPN. Both are secure but Personal is more easily blocked by governments or corporations as they can usually detect the protocol.

Witopia SSL Service: Tunnelblick then Viscosity

Witopia's upgraded SSL service was always a bit of a crapshoot. For years you had to fight Tunnelblick (one of the nastiest pieces of open source software out there, which requires advanced networking knowledge modifying a text file to get anything done: it defaults to not working and makes you move text files all over your computer, authorizing them each time you do). Then came Viscosity which worked a whole lot better. With your Witopia VPN you got a free preconfigured Viscosity client.

Then something went wrong with Viscosity. When you update Viscosity to keep up with Mac OS X, it's another crapshoot if your settings will carry over to the next version.

So as someone who tried to buy three licenses last year (Personal x 2, Pro x 1) for my company and ran in circles for weeks with absolutely silly suggestions from Witopia on how to fix my configuration, I can tell you the dream was over, the shine off of the hood. With the obliging help of Witopia's owner Bill Bullock finally we managed to get a single one of the personal accounts up and limping. Our ideas of using VPN regularly in the various departments at Foliovision went down the tubes.

We cancelled the other two and got on with our lives. OpenVPN on a small VPS turned out to be an even bigger catastrophe. After a whole day of programmer time settting it up, we were able to eke out 2 KB/sec performance. When Witopia is working, you are looking at anything between 1 MB/sec and 8 MB/sec bandwidth up and down. It's not the 100 MB/sec down and the 10 MB/sec up we have on our connection but it's not 2 KB/sec.

So with clients to serve and lives to live, we more or less didn't use VPN except in emergency.

For reasons of my own, I was using VPN via Witopia a fair amount in the last week. All was well until my VPN account wouldn't connect today.

Witopia happily enough has live chat support now. I lost hours with them today so you don't have to.

Why did my Witopia VPN die?

Witopia created a customer portal and consolidated everyone's account history and orders.

When they consolidated the orders some accounts clearly died. Including one of mine as it was a courtesy account offered for last year's nightmare. Tip: don't kill your courtesy accounts early.

Witopia's ability to consolidate our orders years later begs the question how private were our accounts in the first place if Witopia could consolidate them after the fact.

Locked out of the Witopia customer portal

The new portal sounded great. One problem: it was impossible for me to log in with my old username and password. No problem.

I'll reset my password. No luck: No such username.

Joe suggested I reregister. No problem. Wait yes there is, Username already exists.

Joe suggested I create another account using another email and then consolidate them. That sounds like a lot of work and lots of chances for misunderstandings and broken accounts. No, I want access to my existing account.

Joe needs his supervisor.

Tara comes online (last year Tara was the queen of alternative protocols who led me around an enormous emerald coloured garden of irregularly working VPN). Tara remembers me. A bit awkward. Like an affair gone wrong. Happily she doesn't hold a grudge and we get to work.

Fortunately, Tara is able to get a link which allows you to login to your account and reset your password. Remember Witopia is not access to your data. Witopia is only access to your VPN accounts, so security is important but not paramount. If someone sneaks into your Witopia account, the most they can do is cancel your VPN without asking or change your passwords on you or use your VPN surreptitiously for their own purposes (actually that could get you in trouble if they did illegal things while using your account: but the same applies to your home broadband connection).

Inside the new Witopia Customer Portal

So now I'm in the portal. There are all of my orders for the last three years. Hurray.

On the two active orders, there's 412 days left on one and 46 days on the other. The other is also listed as cancelled, with no options for checking data usage, resetting password or assigning the VPN to someone else.

Looks like portal consolidation this week killed off the courtesy account. Sloppy programming. Thanks sloppy Witopia programmers, you've just stolen two hours plus of my life getting all this working again.

Tara asks me to install Witopia software and use that to access my VPN. No dice. With this track record on working software, there's no way I'm letting Witopia's direct installer get at my network settings. I have work to do this week.

Remember Tara loves the alternative protocols. So I set up all of the different OS X VPN protocols following her instructions.

Alas no protocol, PPTP, ipSEC, l2tp will work.

Testing the main Witopia account

I give Tara the password for my main account (not the courtesy one) to test herself. Tara disappears for about seven minutes. When she comes back she announces that the account works just fine and it surely must be a local problem on my end.

I try to reconnect with a couple of the protocols I'd previously set up and tested unsuccessfully. Bingo, I'm on.

"See," admonishes Tara, "it was a local problem after all."

Yes, Tara, but I haven't changed anything in my settings. All that happened was you went and worked on my account with a tech.

Whatever Tara and the tech did while she was away did manage to reset my Witopia account and get it working.

Witopia Speeds

The second test was on an account with send all connections over VPN so I lost my connection to Tara.

Taking the occasion to test while logging on an on, I ran a battery of tests using SpeedTest.net which is nice enough to give both ping and transfer speeds.

Here's what I found:

ipsec New York
ping 111 ms
download 1.2 Mbps
upload 1.1 Mbps

l2tp New York with built-in Mac client
ping 109 ms
download 2.66 Mbps
upload 7.23 Mbps

PPTP New York
ping 110ms
download 9.7 Mbps
upload 7.36 Mbps

Witopia VPN Software

I was feeling optimistic after seeing all of these protocols work so I decided to give Witopia's custom built software a chance. The download is quick and the installer opens up automatically.

Witopia's software gives a nice blue icon like airport in the menu bar. It takes up less space than Apple's built-in VPN. I tried the built-in L2TP and got some surprising results.

L2TP New York with Witopia client
ping 114 ms
download 8.43 Mbps
upload 6.86 Mbps

I'm still using this connection.

Here's what the interface of the Witopia VPN client looks like:

WiTopia application interface extras

Witopia application interface

Using either built-in or Witopia client software is fine. If I had to do it again, I'd probably just stick with OS X's built in protocols. I'd use PPTP as it tests out very fast. If you want to use a lot of locations, then Witopia's VPN software might be for you as you won't have to build all the locations by hand.

Round two: trying to get OpenVPN up and running with alternative Witopia account

When I got back to my chat, Tara had gone home. A gentleman named Shirin had taken her place. We spent a lot of time trying to resurrect the dead account.

More or less hopeless.

The main OpenVPN connection is very fussy. It wouldn't work with either the Witopia VPN client nor with Viscosity.

Once installed, to reinstall it you need to remove several .kext via terminal. Viscosity which did work at one point, won't anymore. The updater won't update Viscosity and you have to manually find and reinstall all the certificates.

What finally worked was uninstalling launch2netpremium and then doing a hand search for all of its files (a nasty respawn filled networking logs to the point that it was impossible to tell what was happening with Witopia). Next step was to install and reinstall the Witopia client with reboots.

The whole affair required three reboots to work and deleting kexts. You don't want to do this. Viscosity still won't work but for now with the Witopia client I can get on OpenVPN SSL internet.

I spent over three work hours painfully troubleshooting this by hand. Effectively Witopia stole $500 from my company while I did this unpaid work. All I got out of it was this article. Don't you be so foolish.

To be honest, I would not recommend a Witopia Pro account. The amount of time you will spend fighting with software and with cryptic errors just will not justify the small increase in speed and security which OpenVPN brings.

Not only that but a WiTopia Personal VPN account is just $40 (renewal) or $50 new account, while a Pro VPN account is $70/year and will cause you to lose hours with unnecessary tech support.

If you are in China or Iran or some place where you really need OpenVPN, go ahead and spring for it. But be prepared to spend a lot of time troubleshooting.

Resources

Here's a nice historic comparison of Witopia and HotSpotVPN with interviews with both company owners, Bill Bullock and Glynn Taylor.

Here's why you might want a VPN.

Answer number one: you are travelling and need to be sure your web access is not being logged (at least only by your VPN provider).

Answer number two: you need access to another server from a fixed and known IP. It's possible with a VPN.

Answer number three: you want to watch Hulu or listen to Mog or Spotify while outside the United States. If you don't use Facebook, your one choice is Mog.

Answer number four: you need to sign up for some papers or service from outside your home country. I wasn't able to complete a birth certificate request from Europe until I came in via VPN and made the order via VPN.

Laptops for Schools: Microsoft Windows, Apple or Linux

A friend and colleague is about to implement a laptop program for 1700 students over in the amazing technology section in ISB (International School of Bangkok). We are talking about true one to one where each student gets a laptop to take home.

ISB haven't decided whether to put Microsoft or Apple onto their new laptops (actually with Apple you have to buy Apple laptops).

For a smaller program without dedicated IT help, I'd say Apple would be the better choice. But at 1700 students you have the scale to make other choices.

In terms of OS, I wouldn’t choose either Apple or Microsoft. I’d look to the future and give the children Linux laptops. Linux Mint Debian Edition is a very nice distribution which can be based on the very reliable and spritely Debian core (rather than the top heavy and sometimes slow to update Ubuntu).

Of course there are driver issues in Linux, but as you have control of the hardware, you can choose recommended hardware with 100% compatibility and effectively achieve the OS/hardware integration which Apple does at a tiny fraction of the price.

I’m a 15 year Apple user with four Apple computers now (have to get rid of a couple actually). We use all three OS at work but will be moving to Linux over the next years.

Why drop Apple?

Apple is going back to walled garden:

• software: only apps from their store
• data: iCloud for all your data
• hardware: all your devices and all your peripherals have to come from Apple (new proprietary data transfer and monitor connections)

This is an Orwellian world which I wouldn’t want to push children into.

The arguments against Microsoft (backdoors, security issues, shovelware on delivery, performance deterioration over time) have been covered many times.

Let the kids learn how to use real computers where there is a chance to look under the hoods and tinker. It will help them to develop clearer and deeper thinking about IT and technology.

And it will save a boat load of money over 1700 laptops. I'd also try to pick laptops for which I could replacement parts at reasonable prices. I'm not sure what vendor offers that. I know Apple parts are very expensive.

One could still make an argument for hardware quality in favor of Apple in buying the least expensive Apple laptops (say MacBook Air 11" with max memory) and putting Linux on them. Where that gets difficult is you do need a distribution which handles power management on those specific laptops well.

Those MacBook Airs are very light (I have one) and easy to carry around. The maximum memory at 4GB is a limitation but one that young students could probably live with, just as students are forced to live in shared dormitories and only get apartments later in life.

The equivalent of a MacBook Air 11" was only made by Sony a few years ago and cost upwards of $2300. Current US educational pricing on a Macbook Air 11.6" with 1.6 GHz processor, 4 GB memory and a 128 GB flash drive with Apple care is $1332 (bare bones 2 GB/64 GB is at $949 but it's seriously underpowered and nothing is replaceable). Not sure about Thai pricing on Apple computers. The advantage here is that the maintenance for three years would be all Apple's problem.

No, but even the Air doesn't scale out for educational use. The cost for 1700 machines would be $2,260,000.

Steve Jobs holding a MacBook Air: Laptops for Students should be small and light
durable and attractive. They also should not cost $1300+ with reasonable
memory and a multiyear warranty. Photo by MacMessiah.

Does anyone have any suggestions for similarly durable and attractive and lightweight mid-range hardware which would suit Debian Linux (with 1700 laptops, you could even afford to commission some power management rewrites from core Debian team)?

Ivan Pope the Scrivener: Hit and Run Rip Off Artist

There's only one person who hasn't paid Foliovision for our work.

And his name is Ivan Pope. He runs a site called Magazero that we moved from Typepad to WordPress. The move went very well, but Ivan Pope went AWOL after the move.

Ivan Pope is the reason we don't split payments anymore on technical moves.

ivan pope teapot head

Keep reading Ivan Pope the Scrivener: Hit and Run Rip Off Artist

Thanksgiving: Making a Better World

For many years I've admired my sister Julie's charity activities and JKT charity pages on her website. Gradually over time, we've ended up doing a fair amount ourselves.  The last couple of years I've promised myself to put up a list of our activities too. What you believe in is so important.

I was recently at a Canadian round table in Slovakia. The Canadian Ambassador to Prague (Canada for inexplicable reasons does not have an Ambassador in Slovakia though most African republics do) asked us very politely what motivated us to come to Slovakia. One well-known business figure quickly answered, "Money is the name of the game. That's what brought me here and that's what gets us up very morning. Isn't it?" His pudgy face beamed at all of us for certain affirmation.

I nearly choked on my drink. I couldn't believe that anyone could live with such shallow values.

Money is certainly not what gets me up every day. Building a better world is. Sometimes it starts by making sure our team is well fed, well paid and well lodged. But the true journey goes much farther.

Cancer survivor Julie Kinnear leading a huge team in a ride against cancer

I have been somewhat concerned that some of our clients or potential clients would shun our services if they knew what progressive organisations we support. But at the end of the day, a progressive stance is part of who we are. We haven't had too many gun-loving, invasion-happy oppressor clients.

And that's just fine with me.

It's been great working with our clients who are usually small to medium sized organisations (not always: we've had Microsoft Seattle and The Hollywood Reporter among other large clients) trying to improve their own communities.

On the occasion of American Thanksgiving (Canadian Thanksgiving is in October), I finally had  peace from our clients to put up a first draft of an about page with some of the organisations we support. The normal pace of my email around 250 emails/day which I actually have to read and answer about a third: on American thanksgiving it's dropped to under 50: if only Thanksgiving came more often.

I'm especially proud to support Captain Paul Watson of The Sea Shepherd every month and for the work we've done to help Mario Radačovský found an international level modern dance company here, Balet Bratislava.

Both men are doing heroic work at great personal sacrifice. This is really change we can believe in. People who are willing to put themselves on the line to do something for the greater good.

Keep on doing it Captain Watson, ballet founder Radačovský, Professor Juan Cole. Your personal contribution to making a better world is what makes hope possible.

sea shepard paul watson takes a bullet saving a whale

Thank you to all the volunteers at EFF.org, Wikileaks, Series 8:08, Znet for your great work against tall odds.

Thank you to the open source projects and coders with whom we've been able to give something back to the giants like Tim Berners-Lee and Richard Stallman who made this explosion of shared knowledge possible.

Thank you to our clients who trust us with their projects and their dreams (quite rightly it turns out) and give us the opportunity to contribute to their lives and the ability to contribute to the organisations we support.

Thank you to the Anima which supports us and allows us still to toil for a better life for humankind and all beings on this earth. However dark the clouds overhead, there is still light on the horizon.

People who buy iPhones are image-conscious fad-following idiots

“people who buy iPhones are image-conscious fad-following idiots”.

The words of Apple pundit John Gruber of Daring Fireball fame, not mine. But a pretty good summary of the situation.

Gruber was complaining about the brilliant Samsung Galaxy S II ad making the rounds. Here is the long version (1m25s) which you might otherwise miss. There's lots of additional clever repartee not in the airplay version: "I guess this is what adultery feels like," says one of the Apple fans in the queue with the Samsung Galaxy in his hands.

long form version of the brilliant Samsung ad

I'm one of the people who moved from iPhone to Android and is really happy about it. Here's why. I owned an iPhone 3GS. After the initial thrill of ownership wore off, I became very tired of:

• being forced to update to the latest version of iTunes every week
• having my mobile phone tied to my credit card and personal account at Apple, sending all the info in my mobile phone to Apple anytime Apple chooses
• fighting with a virtual keyboard which fills most of the screen when you are using it
• really slow network switching (I live on the border between Slovakia and Austria and need to switch networks often), usually requiring turning the iPhone on and off
• having to hack the iPhone to be able to share the internet connection from the iPhone even to a Mac: and then to be worried that any given update could kill my tethering set up
• looking at really lousy photographs, worse than my two year old Nokias

Keep reading People who buy iPhones are image-conscious fad-following idiots

WordPress images uploaded by client too large? try Imsanity

We have a few clients with teams of hundreds of writers and contributors. It's a real task to make sure nobody uploads 8 MB 4200 x 3700 images to their website.

imsanity plugin

When they do, a couple of things happen: the site slows down. Occasionally somebody manages to put the original in the post and their bandwidth goes through the roof.

While it's possible to put a hard upload limit in which defeats the heavy uploads, the errors create a lot of unnecessary administrative overhead.

Imagine that someone created a plugin which would resize huge images to the maximum size used on your site.

Well someone just did.

Try verysimple's Imsanity.

Weird but very cool branding.

Textile: How to write in the world’s greatest markup language

What is Textile? In the beginning there was Textpattern, Dean Allen's lightweight CMS primarily for weblogs. Textpattern was a direct competitor to Moveable Type, B2 (later WordPress) and Joomla! Textpattern lost the CMS wars for two reasons: SEO unfriendly URLs (required numbered ID's) and Dean Allen's collapse from overwork in 2006 (the story is gone from textism.com and Textpattern is back but still with numbers in the URLs). The WordPress story you all know.

Where is Textile now? Most importantly Textile is the markup language for 37 Signals Basecamp messages and Writeboards and Backpackit.

There's nothing nicer than Textile for making a quick list with asterisks:

* item one
* item two
* item three

becomes

• item one
• item two
• item three

in no time.

Blockquotes are easy to do too: bq. gets you there.

h1. h2. h3. h4. will get you headers of various sizes.

# will get you numbered lists instead of bulleted lists.

I often have to introduce our team to Textile formatting and our clients. That's the essential above. Rather than create too much documentation when there's lots of other great Textile documentation out there. I'll link to the rest.

Essentials of Textile and Basecamp formatting attractively presented by Alex Kendrick.

Great Textile test bed with reference in left hand margin from ThresholdState.com

Not very attractive but covers the essentials clearly: Warpedvisions's Textile Cheatsheet.

Total overkill is ThresholdState's Textile Reference Manual. Recommended only to hardcore geeks.

HighriseHQ issues: Why can’t we create a new Deal from dashboard?

Having trouble creating new Deals quickly and easily in Highrise. I'd like to be able to create new Deals directly from the dashboard the same way you can with Cases.

missing create new deal highrise

37 Signals minimalist philosophy in terms of software is something I really get and really appreciate.

37signals got it so right with allowing us to create a new case from the Highrise dashboard when filing an item.

On the other hand, I don't understand how 37signals can cripple the dashboard File option for Deals by not allowing us to create a new Deal and file the item directly.

Such an easy fix and we've been waiting years.

Unfortunately, this fix will be more difficult to add to our GTD product for Basecamp AscentList than search ordered by date and some of the other nice tweaks coming. AscentList will be free until at least May for those who sign up now, so if you are missing some advanced task and project management capabilities in Basecamp, give AscentList a try. Early adopters will get a permanent discount.

Textile Editing on OS X: BBEdit, iTextile, MarkMyWords

As regular readers know we are heavy users of Basecamp. This month is the first time in a while I'm not happy about our subscription as we've had to move up to the Elite Suite at $249 month as we've run out of Basecamp projects at 100 (we've been rotating them for awhile but five more projects came in and there just isn't space). For those who are counting, that's $3000/year for a software subscription.

Basecamp uses Textile as the main editor (well 37signals have added some kind of WYSIWYG editor lately but for those of us writing messages, comments and writeboards for the last seven years, Textile is in our blood).

So what I want to be able to do is write all my posts and drafts in Textile and then convert them to html for publication (saving the original in Textile for further editing).

For a while I used iTextile a wrapper around a Python script. It worked pretty well but was kind of ugly and not customizable. I gave up on iTextile due to ergonomics. When I went to fire it up again yesterday, it turns out iTextile is PPC only and requires Rosetta. On my most recent machines, I've managed to get rid of Rosetta completely so I was warned about installing Rosetta. I'd prefer not to have the emulator overhead hanging around waiting to steal memory and cycles, so I said no.

There is an interesting application called MarkMyWords from xelaton.com in Germany. MarkMyWords allows you to write in the mark up language of your choice (important ones include Markdown, Textile, BBcode and Wiki syntax) and get html out on the other end.

Preview is live which is very cool.

MarkMyWords does what it promises very well and even includes full screen and distraction free modes. If you are looking for a new text editor, MarkMyWords has a lot to recommend it.

MarkMyWords edit window

MarkMyWords Downsides:

• MarkMyWords is another application to install and maintain and learn across all your computers
• MarkMyWords requires a change in workflow (I write mainly in BBEdit and other people have their own text editor prefernces)
• The icon is fussy and ugly.

MarkMyWords icon

Textile Editing on BBEdit

At this point, I was thinking what I really need is to get Textile into BBEdit. I don't know why the BareBones guys have been so lazy about adding a Textile module themselves. Apparently there's been Markdown syntax for a long time.

I found a reasonably good article about how to add Textile to BBEdit but the explanations aren't very clear and one of the download links is broken and the other doesn't give the right filename when unpacked. [Update: dpkendal's original version was broken - our own Martin Vicenik has fixed it for you and uploaded it.]

So for non-programmers, here's how to get Textile editing working on BBEdit:

1. download our Textil.sh filter from Github.
2. unpack the very long file gist1348479-0d1929ba5ff2b3e2b4293dd63254604b72d62b58.tar
3. you will get a folder with a file with this name in it: "Textile.sh"
   

   Note:If the github ever disappears, here's a local copy. We found that this script has some bugs in it's current version (the constants are not properly added). Before this gets submited to Github.com download the fixed version here: Textile.sh.zip
   Our version also won't convert single and double quotes to HTML entities. This should be an option in the original version, hopefully our changes get into Github.com soon.
   

4. move this file to /Users/~/Library/Application Support/BBEdit/Unix Support/Unix Filters/
5. go ahead and write some Textile
6. open up the Unix Filters palette: Windows -> Palettes -> Unix Filters
7. you should see Textile.sh at the bottom
8. when you are ready to convert your Textile to html, just doubleclick the Textile.sh item. You can create a hot key as well (very useful)
9. as it's BBEdit you can see your html and get a preview of it and then just use undo (command-Z) to get back to the Textile version for further editing
10. when you save your file, make sure you save the textile version
11. for bonus points before posting into WordPress or even Basecamp run the html optimize filter on the result to get rid of all line breaks: Markup -> Utilities -> Optimize

BBEdit unix filters palette

Bingo, you now have full Textile writing inside of BBEdit at zero cost. Apparently this filter will work for other text editors which accept php filters (TextMate among others) but I can't provide step by step instructions as BBEdit 8.7.2 is my weapon of choice.

I may still buy MarkMyWords as I have something of a fetish for text and html editors (own at least ten of them) and earn my living writing and coding. $25 for another work tool is no big deal. But I wouldn't encourage it's adoption across our company as that would be $200 for what most people wouldn't use nearly enough. Our programmers will be much happier with a working php script. On the other hand, Textile.sh doesn't require me to change my workflow at all.

This article full of ordered and unordered lists was written in BBEdit and Textile.sh with no issues.

Marked does not get a review here as Marked is AppStore only. I will not sign into or buy anything from the AppStore or even let it run on my computers (the AppStore is effectively a back door).

Searchlove 2011 Foundation Bar 24 October

Search Love Conference London 2011 – Day Two

Full set of photos of all searchlove 2011 speakers from day two here.

Despite potential hangover from the last nights party, everybody came thrilled to the second day of Search Love Conference in London, co-organized by SEOmoz and Distilled.

The first runner was Ciarán Norris of Mindshare. His lecture Personalization, Profiles and Privacy didn't offer many calls to action, but two of them were important for all marketers:

Personalization, Profiles and Privacy presented by Ciarán Norris

• Web is becoming more and more personalized. Get ready for it, you will have to learn more about visitors and treat them individually
• There are the Web rules, but there are also the legal rules of the country you operate in. If you are tracking your visitors, you may get burnt.

Keep reading Search Love Conference London 2011 – Day Two

SearchLove Conference London 2011 – Day One

SEO and social web conferences organized by SEOmoz and Distilled gained high reputation as major events in web search industry, and Foliovision is glad to take part once again. London Search Love 2011 was slightly rebranded, but that should not confuse you – linkbuilding and social media still play the key role!

SearchLove Congress Centre

Keep reading SearchLove Conference London 2011 - Day One

WordPress SEO plugins review: All in One SEO, Platinum, Yoast compared

There are a lot of SEO plugins for WordPress. "SEO for WordPress" has become a major catch phrase. Many companies now offer paid plugins which provide a lot of different options and possibilities how to further customize your site SEO to the tiniest details.

How much of it is really necessary or even helpful? Are the basic page title and meta description fields not enough? Should you pay up to $47/site/month for a plugin only to be able to set "noindex" for some of your categories? Will it help you to spend a week of sleepless nights learning what "noindex" is and when it might be helpful or are there other things you might do during that week which might do more to improve your rankings than "noindex".

Here's some alternative suggestions:

• write the four or five great articles that you've been meaning to do for months
• go out and comment up a storm on the most interesting weblogs in your sector with a link back to your site. You won't get much Google juice at point but you'll bring new visitors.
• answer some questions on related forums to the point people trust you and visit your website for additional information

We believe that every site owner and almost every author on a site should be able to do the basic SEO optimization of his or her own site. This means the SEO plugin should be easy to use and have good default options, which won't put the site at risk. Some plugins risk seriously compromising your indexation in Google just by activating the plugin. In Foliovision's opinion, this is totally unacceptable. We believe in safe intelligent defaults. There's no reason the site owner should have to worry about his or her SEO plugin.

Plugins compared in this article:

• All in One SEO Pack
• Platinum SEO Pack
• WordPress SEO by Yoast
• wpSEO
• Built in SEO functions of Thesis template
• Built in SEO functions of WooThemes - Premium WordPress Templates (we used Daily Edition)
• FV Simpler SEO

Some explanation for the terms we use in the comparison tables:

• Post - any post type, that means post, page or any custom type
• Taxonomy - any kind of taxonomy - category, tag and all the custom ones
• Section - section of a website - front page, category archive page, monthly archive page and so on

General information about tested plugins

Let's start by overview of the basic plugin attributes:

Conclusion

Interface - The vertical space on websites is precious and it also applies to the WordPress post editing screens. Why some of these plugins use so huge interface boxes, that they took over nearly all of my 1680x1050 screen (Thesis is the worst, others just like to waste a lot of space on large padding)?

Compatibility - SEO functions should not be part of any template. What happens when you need to change your site design? You waste extra time on trying to move your existing SEO data from Thesis into some reasonable plugin.

Branding - Most of the plugins put in their branding in your site HTML code. I don't understand why all the SEO plugins have to be like this, I've never seen that anywhere else! Imagine that you have a site with 20 plugins and now each of them suddenly decides that it's going to put some HTML code in the header to let everybody know what you use this or that great plugin by Mr. XY.

Basic functionality

The basic purpose of a SEO plugin is to allow easy editing of all the extra information which gets indexed by search engines:

• Title tag
  • this might be different than the post title in WordPress
  • so it's a nice way how to put in some extra keywords
• Meta Description
  • it's typically displayed bellow each link when you list search results
  • hand written descriptions are the best
  • Google shows meta description based on what keywords are using in the search query, so don't panic if you don't see your descriptions all the time
• Keywords
  • don't have so much power as they used to

The comparison table of basic SEO functions

Conclusion

Title - most plugins require title rewrite enabled to start using the field for SEO title. Title rewrite is by default "on" in most of the plugins, so what happens when you don't want to use it and you are happy with you titles as they are without the plugin? You get no custom SEO titles, or you have to tweak the rewrite to not change the structure of your titles.

Desciptions - the only thing what really matters here is the hand-written description. You don't need to auto-generate that, as search engines will do so by themselves.

Keywords - are not considered by search engines so much anymore.

SEO editing fields for custom post types are missing in Thesis and WooThemes Dail Edition.

Advanced SEO functions

Most of the plugins contain some additional options which might be and might be not helpful for your site. 

• Noindex
  • tells search engines not to put the webpage into its index
• Nofollow
  • tells search engines that the link should not influence search engine rankings of the target
• Noarchive
  • prevents content from being stored in search engine cache
• Canonical link
  • sets the URL under which the webpage should be indexed in search engines
  • WordPress takes care of this on its own for posts and pages
• Directory tags
  
  • allows you to block use of DMOZ open directory data instead of your descriptions

Conclusion

Thesis offers some crazy anti-SEO options, like making sure your front page won't get indexed. That's really a dangerous function, imagine somebody turns it on by mistake and you find out about it few months later. Your search engine rankings would probably be way lower than they used to be. If somebody is doing really a special site which needs to have noindex on front page, surely he can do it without Thesis.

WordPress SEO by Yoast offers a nice SERP preview feature, but it doesn't respect the character limits on title and description. It also has many different features, perhaps more than a SEO plugin should handle. I imagine it's hard to switch to another plugin later is you already like the provided breadcrumbs or XML sitemaps.

Miscellaneous

Here's little extra from our experience. Reasonable Default Options are one of the reasons why we started our own SEO plugin. We were not happy to tweak the settings (or plugin files) on each site over and over again.

Conclusion

Default options - When you install a SEO plugin, make sure you check how the default options change your titles. With FV Simpler SEO, you don't have to be afraid, as there is no change unless you rewrite the title yourself.

Watch out for WordPress SEO, it relies on the template to use the most correct function to display the meta title tag, which is not the case of the standard WordPress template (it uses some extra echo statements in the HTML of title tag). However - the plugin should be able to handle that without destroying the titles in the first place.

WooThemes Daily Edition put's nofollow into header of your posts by deafault. This can be turned into follow in options or changed for individual posts (not working on our test site with clean fresh WP 3.1.1.) We recently had a client install a WooTheme and deindex his site. Needless to say we helped him out of the bind but he was unnecessarily freaked out. Bad choice Woo. But then based on hard experience we don't like Woo Themes or most other commercial WordPress themes at all.

Which one is the best?

Depends. If you like to play around with WordPress and you are interested in the most finest SEO tweaks, then check out WordPress SEO by Yoast. If you are a writer, or you don't want to worry about your SEO plugin deindexing you by accident, then our FV Simpler SEO is the safest and easiest to use by far.

If you think otherwise or want to recommend another plugin, we'd love to hear what you like about the other plugin and why. If you have any suggestions on how to improve FV Simpler SEO, let us know.

Balancing life and work as an entrepreneur

Lavalife was a sleazy enough venture but Bruce Croxon did manage to sell it for $140 million.

CBC asked him about life/work balance for entrepreneurs:

What's your best advice for balancing life and work as an entrepreneur?

Forget it.

Good advice.