vBulletin Tagging

Wow it’s really been 1 year since Gravatar joined the Automattic family.  Time sure flies when you’re having fun!

Gravatar has come a long way from the service that it was back in October 2007.  My wife likes to laugh at me because I’ll pick something up in one room, like a remote control, and move around the house fidgeting with it.  Then I’ll absentmindedly leave it in some random place like the bathroom, or the freezer.  One year ago we picked up a small struggling avatar service with a great name and an awesome fan base.  Now, in an attempt not to leave it in a random location, we’re looking back on the last year (and letting you look with us.)

The service was running version 2.0, and set up on two rented (or collocated, I don’t honestly know) servers.  The servers were running at loads of around 20, and could spike to well over 100 (that’s a lot.)  It was obvious that we needed both some stop-gap fixes and a plan. The first thing we did was throw some caching servers in front of the service — a couple of varnish servers as I recall.  This dropped the workload of the two boxes considerably, and allowed us to look at Gravatar without bringing the service down.  Next we replicated the setup 1 for 1 to two of our own (more powerful) servers.  This gave us a bit more breathing room.  And we began to plan.

It was obvious, from the very beginning that the service was going to have to handle a constant torrent of requests from the internet.  Most of those requests would be for email addresses with no Gravatar and come from URLs that could be crafted in an unlimited number of ways.  On top of that we knew that we wanted to make all the paid features free, and expand the size a Gravatar could be from 80 pixels to 512 pixels.  So basically instead of our goals being to make the undertaking less daunting we we’re actively making it a more intense challenge than it could have been.  But that was OK with us, because our goals for Gravatar weren’t to make it easier but to make it better.  We wanted to make Gravatar the kind of free service that we could use, would want to use, and would be proud to share with the world.  I know that last bit sounds like marketing crap, but that’s really what we wanted to do and is really how we look at Gravatar.

Pretty much the next thing we did was port Gravatars code from RoR (Ruby on Rails) to PHP.  As I mentioned when we announced this change the reason for this wasn’t about Ruby or Rails.  Simply put we’re a PHP shop, and once rewritten in PHP we have many more great minds that we can easily throw at it than if it were still in RoR. Since we ported it *pretty much* directly from rails there are some left-over rails-isms in Gravatars code that you wont find in, say, WordPress. Shh…. Don’t tell Matt

In the rewriting we tried to tackle the largest scalability problems with the design of the service.  You can imagine that for an avatar serving service… storing, searching, and serving avatars is paramount.  Gravatar 2.0 (pre PHP) suffered from some pretty significant inefficiencies in this regard, and I think that a big part of that was limited resources (time and servers.)  Luckily we we’re now not significantly limited by either of those things.

– warning beginning technical details which may be safely skipped over if you don’t care –

The way that images were stored originally was: a complete image was made for all sizes between 1×1 and 80×80 pixels, a directory made for each rating, and a symlink placed from the rating to the appropriate image (either the users image or the default image in case the rating was too high.)  So that’s 80 images, 5 directories, and 240 symbolic links.  The reason for this, I believe, was to attempt to serve the avatar content without any database interaction whatever.  The files were then archived, uploaded to Amazon S3, and an entry added to Amazon SQS.  Finally the SQS entry was retrieved by the serving server, the file downloaded, extracted, and placed on the filesystem.  So this is why it took several minutes once you uploaded and cropped your image for you to be able to browse the rest of the site again.  You can imagine how many files Gravatar was comprised of by the time we got a hold of it!  We knew that this would simply NOT work for our new 512×512px avatar sizes.  Lastly there were a couple of directories which had several hundred thousand entries (either files or other directories) which were nearly impossible to even get a listing inside of.  So we had a list of things NOT to do.  We just needed to figure out what TO do

So we decided that we would render all our avatars dynamically from the highest quality copy of the image we can manage… down.  We would only store one version of the image, though we would store it in multiple places (a local file server for speed, and S3 for redundancy.)  We would still rely heavily on caching.  And we would asynchronize as much of the workload as was possible, so that you don’t have to wait for things to happen after you finish cropping (to do this we employed various techniques and hacks best left for another day and another story.)

– ok this batch of details has been concluded –

So the problems were many, one year ago, and the challenges were fascinating.  I recall being overwhelmed by support requests for quite some time.  I would get 40 emails on a good day, more on a bad.  And believe it or not your emails very much shaped the future of Gravatar.  I would group them into specific problems, and always fix at least the largest problem (volume wise) each week.  Over time the service has grown quite stable, support requests have gone down to just a handful every day, and things are generally peppier than ever.

We had some some bumps tuning our caches… for a while there we accidentally told your web browsers never EVER to re-validate an image. But we got that handled in short order… and things are zipping along quite nicely.

Gravatar now lives on about 20 servers. 2 Database servers, 1 File server, 2 Load balancers, 5 Caching servers, 9 Web servers, and 1 Development server.  That combination of servers is handling an average of 7,214 of your requests every second of every day.  That’s a whopping 623,293,056 requests each and every day! 96% of all of those requests are served directly from cache.  These days we get around 5,000 uploaded images every day.  Even with this staggering increase in the number of requests we’ve been able to make Gravatar faster, and more reliable than it’s ever been.

So here we are, one year later, looking out over the vast frontier of the internet and contemplating the future of Gravatar.  There are a great many things that it could become.  We know that we don’t want to loose focus on the core of the project: Serving your avatars (that’s what it’s all about!)

We know that an avatar is “a graphical image that represents a person, as on the Internet,”  But it’s also “an embodiment or personification, as of a principle, attitude, or view of life.”  And that is exactly where we are headed:  Making Gravatar a place where you can do more than just store an image, making it a place that can be your presence online.  So we’ll be rolling out more features in the near future to allow you to store more data inside Gravatar — and more importantly to allow you to use that information in other places on the internet through open standards.

We hope that you’ve had as awesome a time using your Gravatars as we’ve had making it all work.  And we look forward to the future — to when your identity doesn’t have to be cemented to a specific site, but is fluid and flexible, and persistent.  We hope to see you there!

Cheers!
The Gravatar Team.

I’ve just released the latest Blog Herald WordPress Wednesday News on WordPress Wednesday News: WordPress 2.7 Final Features, WTC Plugin Contest Winners, WordCamps, Surveys, and More, and there are some WordPress news tidbits I’d like to call your attention to.

WordPress 2.7 Feature List Set

Ryan Boren has announced the final WordPress 2.7 features to prepare us all for the upcoming release.

They include the new “crazyhorse” WordPress Administration Panels interface, a new Write Post Panel that allows dragging and dropping of meta boxes so you can rearrange your writing surface, the ability to hide columns on the content index pages, inline editing of posts and Pages on the new content index pages, comments XMLRPC API, sticky posts, automatic WordPress Plugin installation and integrated Plugin browser, HTTPOnly auth cookies, and much more.

After all the kvetching I’ve done about the Comments Panel in the WordPress Administration Panels (okay, I’m not the only one), they have finally overhauled it and have added the ability to automatically upgrade WordPress and the total revamping of the Comments Panel to include replying to comments, keyboard shortcuts (hot keys) for managing comments, and threaded comments to help you keep track of the conversation.

I’m waiting to try it out as it may soon be added to WordPress.com as part of the testing. I do hope it will be as user-friendly as WordPress Comment Ninja Greasemonkey Script by Engtech of Internet Duct Tape. I’ll hate to part with my favorite non-WordPress Plugin toy.

Tell WordPress How You Want the WordPress 2.7 Administration Panels to Look and Work

In announcements on the WordPress Development Blog and on WordPress.com, WordPress users are invited to participate in a survey to have your say on how the navigation options on the WordPress 2.7 Administration Panels interface (UI) should look and work.

At WordCamp 2008, Liz Danzico and Jane Wells presented Riding The Crazyhorse: Future Generation WordPress, an inside look at how the new interface was developed and the amazing technology they use to test the results and make improvements.

If you want your say in how it looks, you better speak up and take the survey.

Weblog Tools Collection Plugin Winners Announced

While WordPress 2.7 is rewiring the whole Comments Panel, it is telling that the grand prize winner of the Weblog Tools Collection WordPress Plugin Competition was WP Comment Remix WordPress Plugin by Pressography. It will be interesting how this incredible WordPress Plugin survives the next release of WordPress. I’m sure the creative Plugin author will come up with something even more interesting.

Second prize went to the author of the Manageable WordPress Plugin, followed by third prize, WP Easy Uploader. Consolation prize goes to Prelovac for his amazing work on WordPress Plugins recently.

All the Plugins in the contest are amazing. Check them out and see how creative the WordPress Community continues to be.

Smashing Overload of WordPress Development Resources

Smashing Magazine published the WordPress Developers’ Toolbox, a huge collection of tips, tricks, resources, WordPress Themes, WordPress Theme design help, documentation, and tutorials for WordPress users. It’s overwhelming but there is a ton of great information in there.

Please make sure that the article you are reading covers the version of WordPress you are using as some of the information is version-specific.

A Ton of WordCamp Information

I continue to report on the many WordCamps and WordPress Meetups that are popping up everywhere. If you want your event covered, make sure to email me so I can cover it on the Blog Herald WordPress Wednesday News and The WordCamp Report.

In case you didn’t hear, The WordCamp Report is looking for volunteers to live blog or report on WordCamp and WordPress events around the world. The site is covering WordCamp news, events, and WordPress events, and includes tips for those who want to bring a WordCamp to their area.

If you have been a part of a WordCamp event and would like to submit an article on your experience and lessons learned, just let me know.

And More WordPress and WordPress.com News

I also covered a lot of other WordPress news including these recent announcements on WordPress.com:

SSL Now Available for WordPress.com Bloggers: WordPress.com announced the ability for their bloggers to protect blogs with SSL. If you are blogging on a public access point, “blogjackers” can prey on security vulnerabilities and possibly collect sensitive information. By adding SSL protection, you will automatically be logged onto your blog with an HTTPS connection (secure). You’ve always had the option, but few WordPress.com bloggers understood how it worked.

To activate this new feature, go to My Account > Edit Profile in the WordPress Administration Panels and check of “Always use HTTPS when visiting Administration pages” under Browser Connection.

WordPress.com FAQ Gets Screencasts: The WordPress.com FAQ now has screencasts on their guide posts helping WordPress.com users learn more about how to use specific WordPress.com features.

WordPress.com August 2008 Stats: The August Wrap-Up for statistics on WordPress.com included 286,860 blogs created, 298,655 new users, 3,442,638 file uploads, 439 terabytes of content transferred from our datacenters, 6,750,960 comments, and 1,196,661 active blogs and 14,056,683 active posts. WordPress.com also crossed four million blogs this past week.

New Themes for WordPress.com Members: WordPress.com announced two new WordPress Themes recently. The Albeo Theme for WordPress.com bloggers is designed by Elena at Design Disease. The DePo Masthead WordPress Theme by Derek Powazek features a minimalist magazine design with dynamic features such as header font size changes depending upon the length of the blog title, column heights for front page articles will always be equal, interesting navigation options, and hidden, unfolding comments.

Site Search Tags: wordpress news, wordcamp, wordpress events, wordpress 2.7, weblog tools collection wordpress plugin, wordpress plugins, wordpress plugin news, wordpress announcements, wordpress help, wordpress tips

Subscribe Via Feedburner Subscribe by Email Visit
Copyright Lorelle VanFossen, the author of Blogging Tips, What Bloggers Won’t Tell You About Blogging.

Hurricane Gustav satellite photo

Jonathan Bailey of Plagiarism Today just called me from Northern Louisiana to report that he and his family have survived the evacuation from their home in Shreveport, Louisiana, though Hurricane Gustav appears to have done more damage where they evacuated to rather than where they left from.

Jonathan and I were both victims of Hurricane Katrina three years ago when that storm ripped our lives apart in its wake, and WordPress played an important role in our lives then as well as now. I was in Mobile, Alabama, at the time, having just started Lorelle on WordPress on the brand new WordPress.com a week or so before the hurricane hit the Gulf Coast, causing destruction over tens of thousands of square miles along the coast and deep into the country. The hurricane’s storm surge plowed 22 feet of water up the Dog River to where we lived, bringing destruction in it’s wake and retreat. Tornado force winds ripped out trees and turned them into missiles. Nary a sign, building, nor person was safe with the winds and the force of the storm surge waters.

While running from the storm to Atlanta at the last minute when the score turned from Category 4 to Category 5, only to have Katrina follow us and beat us down, causing death and destruction in Atlanta a few days later, I blogged about many of the first blogs about Hurricane Katrina and about how blogs helped and impacted our lives in the wake of the storm. Katrina also impacted bloggers hit by spammer abusing Katrina fascination with their comment spams within a couple weeks of the hurricane’s landfall.

In Blogging in a Disaster on the Blog Herald, Jonathan Bailey reported on how he continues to blog even as an evacuee, maintaining his online work as well as staying in touch and reporting on his own hurricane experience. He offers some great tips, including a new one I wished I’d had during my own disaster experiences. He recommends your site is hosted on the server cloud or grid, which I spoke about recently:

3. Move Data to The Cloud

Any information that you need to run your site, put it in a secure place on the Web. Remember that you will likely be using a strange computer so any URLs that you can’t remember and type by hand, put those on the Web too. If you use a program to store your passwords, store a cache of those on the Web also. Obviously, you’ll want to encrypt and secure such information before posting it.

Even if you have your own browser on a thumb drive, as recommended above, it is best to make sure everything is in multiple places. I have been repeatedly saved by my private bookmarks on Diigo.

Today, we have more resources for backups and protecting our site, so take advantage of them now in safe times, as well as for disaster protection and backups. For those with blogs on the cloud or grid and using cache Plugins and the latest version of WordPress with improved cache, their sites could easily flex and withstand the sudden influx of traffic without crashing or breaking bandwidth restrictions.

In addition to Jonathan’s tips on blogging a disaster, I’d add:

• Bring cameras, video, and audio recording equipment to preserve your memories and experiences for uploading to your blog and for your own records.
• Bring more than plenty of all the appropriate batteries for your various equipment.
• Have your blog setup with image, audio, and video uploading WordPress Plugins installed and be familiar with how they work.
• Sign up in advance with video, audio, and image hosting services so you can quickly upload, store, and link to those images from within your blog.
• Backup your evacuation and escape routes. Have at least two backup plans for evacuation and escape, before evacuating and for during the evacuation. Evacuation routes shift as the storm changes its course, and the storm may suddenly change speed, arriving ahead of schedule or behind. Be prepared to move fast and stay long.

Blogs Offer Communication, Information, and Connections During Disasters

In the days leading up to Hurricane Katrina, blogs served up information and first hand experiences of the wait for the massive hurricane. People provided resources for help, evacuation, and reports on what they were doing to prepare for the hurricane. With the fast and easy set up for a WordPress blog, and the newly created WordPress.com free blog hosting service, many of those Hurricane Katrina blogs were powered by WordPress, as are many blogs with Hurricane Gustav news and reports.

During the hurricane, a few blogs, typically ones supported by news agencies and the media with generators and situated in defensible and protected locations, kept us up-to-date with the disaster as it happened from their perspective, as did the media in general.

I remember watching weather reporters hanging onto railings as debris flew all around them, microphones clutched in their white knuckled hands screaming and shouting to be recorded over the strong winds and crashing water sounds. I thought then as I do now, “We don’t need to have a blow by blow report that bad. We can wait.” But the media needed the sensational perspective to keep watchers glued to their televisions and websites for the up-to-the-minute status of the ent. And some bloggers tried to emulate the journalists, putting their lives at risk.

With uploading and embedding of video with WordPress Plugins like PodPress as well as the Smart YouTube, NextGEN Gallery, Flickr Photo Album, and the popular Viper’s Video Quicktags, it’s fast and simple to add video of all types to your WordPress blog. However, never risk your life for the sake of images for your blog. Leave that to the professionals - whom we hope are better trained and skilled at surviving the hazards mother nature throws their way.

With debris covering the highways and access points, it took a while for rescue and recovery services to get into many areas, as well as for evacuees to return. For those still connected to the web with generators and satellite or still working cell phones, blogs brought information faster than most news and government agencies. They also spread rumors like wild fire.

After the hurricane, blogs played a huge role in not only reporting on the activity, but exaggerating it. In my article series on controversies and blogs, I wrote about how the events in the New Orleans Astrodome were exaggerated and turned into an urban legend due to a blogger faking the story:

Hurricane Katrina brought of a lot conspiracy theories and hoaxes into play, including a huge one which played into the New Orleans Astrodome myth in the wake of the disaster. Since it was easy to believe that people will descend into violence and “ugly” at the drop of a social hat, and that the government is useless in such situations, many believed what was reported as an “eye-witness account” instead of looking at all the information before judging. Supposedly eye witness accounts reported on horrors that were later found to be untrue, and there is a lot of confusion over who actually wrote this and whether or not they were even there. Yet, many bloggers continue to point to these as proof of whatever theory they support on what happened in New Orleans after Hurricane Katrina and the dikes breaking.

We’ll have to watch out for Gustav scams, but in general, blogs did more good than bad as many bloggers stepped up to help by checking out their neighborhoods and communities and reporting on the status of individual homes and businesses, by helping to connect families and friends separated by the storm, by recruiting and directing volunteers and donations for delivery to the needy all along the Gulf Coast, not just to New Orleans. Bloggers pitched in, reported, helped, and aided others all over the country, including opening their homes up to those in need.

Learning from the past, many popular and high traffic blogs jumped up to offer links, resources, and references to help with those confronting or trying to comprehend Hurricane Gustav including HGeekdad from Wired.com, Read/Write/Web, and others.

Social media in general, including microblogs like Twitter, Friendfeed, and Tumblr, played a huge role in preparation, response, and reaction to Hurricane Gustav. According to Weberence.com, Hurricane Gustav is the first major US hurricane covered by microblogging services like Twitter. Within a short time, a specific Twitter Search for Gustav became a popular search and feed. No Turn on Red reported on how Home Depot is using their Twitter account for tips on how to prepare for the upcoming hurricane, one of many using Twitter this way. Ki Mae Heussner of ABC News even covered the impact of social media as an “emergency tool” for reporting on the hurricane.

Legitimate news media used Twitter, too. The New Orleans Times-Picayune (NOLA.com), the USA Today Weather Guys, Reporters from the Chicago Tribune, and even the Red Cross all used Twitter to microblog the news, tips, resources, references, and information.

Jazzy Chad created the Hurricane Gustav Twitter Widget if you wanted to track Hurricane Gustav news on your blog through Twitter.

A FriendFeed Hurricane Room for chatting was also started, incorporating Twitter and Friendfeed and online chatting.

A Facebook page was created called the Hurricane Gustav Digital Support Brigade to help connect those seeking help and information on the storm.

Gustav Information Center on Ning became a major resource for many, including media and government sources. It featured a forum, blog posts, Twitter, news, links, video, photos, and a variety of feeds from all over with information on the hurricane, responses, and services, as well as individual bloggers covering the hurricane on their blogs, much in the style of the upcoming WordPress Theme for social media, BuddyPress. Another Hurricane Gustav aggregator was created to collect information from Twitter, Google, YouTube, Flickr, and more.

Weather bloggers were in their prime, many using Wunderground’s WunderBlogs network to link together their blogs. Dr. Jeff Masters’ Weather Blog, Climate Change Blog, Ultimate Chase Photo Blog, Northeast Weather Blog, Tropical Weather Discussion, StormW’s Tropical Forecast Desk, vortfix, Tazmanian, vortfix, mysticmoondancer, and cyclonebuster had hundreds of comments, the comments often containing more information than their blog posts as many discussed the reality on the ground versus the satellite and weather images and reports and their perceptions of the information posted. The The Weather Nerd and BehindtheWeather.com were recommended by many for coverage of the storm.

Within minutes of the first announcement that Gustav was heading towards landfall, a Wikipedia page for Hurricane Gustav was created and updated through the storm’s course, and will continue to be updated as more information is added as the facts come in.

Today, we have more sources than ever to track what is going on with a disaster as well as a way for individuals to express themselves before, during, and after disasters.

For the bloggers living, working, and surviving in disaster areas, they have a lot to teach us about how blogs can help and serve our online community. Those who want to help from outside the impacted areas are learning more about how to integrate multiple media and blog sources into a single aggregator without impinging upon copyrights, creating central clearing houses for news and information. The more we learn about how useful blogs are in a disaster, the more our blogs will improve overall.

Site Search Tags: weather, weather blogs, disaster, disaster reporting, disaster blogging, hurricane, hurricane katrina, hurricane gustav, gustav, katrina, new orleans, mississippi, gulf coast, jonathan bailey, plagiarism today, blog news, blogging tips, how to blog a disaster, blogging disasters, blog news, news, blogging the news, blogging events

Subscribe Via Feedburner Subscribe by Email Visit
Copyright Lorelle VanFossen, the author of Blogging Tips, What Bloggers Won’t Tell You About Blogging.