So I think it’s about time I give a general update as to where I am with my various personal projects…

Snapr Photo Gallery

I had Snapr running on my local server for quite a while, however it being there meant I couldn’t really show my photos to the world as it depends on my ADSL being stable/fast. This also put a burden on the Internet connection which I can’t really afford since when I’m not coding, I’m usually playing games or hosting games, and to randomly have my upload disappear wasn’t great. So about 8 months ago it got turned off, however, as my holiday is quickly making an appearance I might resurrect it’s code base, but with whats going on at the moment I doubt I’ll be able to get it working again in time.

GOW Take 2

I’ve been meaning to move away from Wordpress for a while, not for any particular reason, but other than I’d like to have coded my own; especially since the closest I’ve gotten to coding a blog in rails was a very simple one for part of a big website. I want it to support all the basics such as trackbacks/pingbacks so that there is no major blog downside from using it over Wordpress. This is coming together quite quickly at the moment, it isn’t going to be a Wordpress clone, as I have no intention to support multiple themes/users. I think I’ll release the code base when its finished, but with no intention of supporting/maintaining it beyond security issues.

I Watched That

This is a project I’ve had on back burners for nearly 2 years now, it’s basically last.fm for the TV world. So you can mark which shows you watch, comment on episodes, and mark specific seasons/episodes as watched so that you can keep track of what your yet to view. This would also include a recommendation/suggestion engine, along with the usual social networking side of things. Initially started on a really old version of Rails, it’s currently being upgraded to the latest Rails 2.3.2. Taking much longer than I initially thought as most of the code I wrote back in the day is well… not so good. So big chunks are being re-written to make them more efficient and to satisfy the part of my brain that’s a perfectionist.

Host Controller

This was originally a XP only utility that I used to use while developing on Windows. It allows you to toggle entries in the Windows host file on and off whenever you like. After I switched to developing on Ubuntu I had no real use for this application, and since then it remained in a working state for XP. Since now I have a Vista machine I decided to give it a little go just to make sure it still works. Had to make some small tweaks to get it operational again, however it’s now in a working state for how my machine is setup. (So chances are, it probably won’t work on your Vista machine). I need to support UAC, which currently it doesn’t. But beyond that, it’s a small cheerful program that does its job. If you want it, you can find the source code on github and a download of the application can be found here.

So quite a quick round up on whats going on, will hopefully post some more around the Rails blog, since that’s what I’m working on at the moment in my free time. A post on default_scope is definitely on the horizon.

An odd titled name for something that I had to remember how to do today. If you need to sort something by the number of its has_many object, eg…

Album has_many photos, listing albums with the most photos first
Forum has_many posts, listing forums with the most posts first
Group has_many users, listing groups with the most users first

…and you want to show these parents ordered by the number of children objects they have you need to do this…

a = Album.find(:all).sort { |x,y| x.photos.size <=> y.photos.size }

Obviously you can use named_scopes and conditions to restrict it down if you don’t want all of a certain object, either child or parent.

Hopefully this helps someone out.

Had to go look this up today as we are using more named_scopes at work now, although couldn’t off the top of my head remember how to get a named_scope to take an argument nicely. Anyway, the solution is…

named_scope :recent, lambda { |*args| {:conditions => ["created_at >= ?", (args.first || 10.minutes.ago)] } }

Hopefully this helps someone out

So yes, I recently got myself a new home server off of eBuyer, and I decided to install Ubuntu on it. After installing various other packages without issue the first to cause problems was Rails (it always would be wouldnt it). Either way, when you try to run…

sudo gem update --system

It goes…

/usr/bin/gem:23: uninitialized constant Gem::GemRunner (NameError)

Which is really useful and does exactly what you want it to.. not. The fix is to go edit the gem file…

/usr/bin/gem

In there, hunt for the line…

require 'rubygems'

…and just after, put this…

require 'rubygems/gem_runner'

Problem solved!

For those who know me (hi people on facebook) you might know that over the past few weeks I have been slowly making a gallery in Rails for my personal photos. This started off the back end of being annoyed at Flickr’s free account only allowing you to create 3 sets (or albums) before you had to pay for more. This annoyed me as I do like the Flickr layout, but I’m not really prepared to pay for it just yet.

So instead, I decided to make my own Flickr clone, and this has been aptly named Snapr!

I started off by looking through their design, seeing how they spaced and laid things out, but one thing in particular that interested me was how pictures seemingly look better on Flickr. Now of course, its not just because of its design that compliments images and that’s enough to make them appear to be better. But as stupid as it sounds, pictures on Flickr do look better, it took some research and testing to finally work out how they do it, but here it is.

After digging through the code in the Mac Flickr Uploader, which is quite handily open source, it was just a case of attempting to follow the code through to work out what they did. They use GraphicsMagick for image manipulation and this is the code that makes images look great…

// Find the sharpen sigma as the website does
double sigma;
if (base <= 800) { sigma = 1.9; }
else if (base <= 1600) { sigma = 2.85; }
else { sigma = 3.8; }

// Create the actual thumbnail
img.scale(dim.str());
img.sharpen(1, sigma);
img.compressType(Magick::NoCompression);
img.write(*thumb_s);

This is the last part of the generation of thumbnails, and as you can see, it scales down the image, then sharpens it by this magical sigma amount. To replicate this I had to actually play around (for quite a while) with that sigma figure so that I could make a like for like replication of their images. Obviously they don’t compress the thumbs, as if you do, they look horrible.

Now initially, this was easily to replicate in Rails with RMagick, however when I switched over to JRuby there isn’t a RMagick gem, which is more than a slight annoyance. After some help from a friend I had my own implementation of pure Java for image manipulation, and this worked fine. However I had been using the attachment_fu plugin to make uploading and resizing of images a breeze. (Read: if you aren’t using attachment_fu, use it!). So the next step, how to get this new Java code to work with attachment_fu. 2 evenings worth of playing and testing later (and my complete lack of any real Java knowledge) I admitted defeat. So, at this point I have a fully working gallery, minus image uploading and processing, which as far as galleries go, meant it pretty much useless. What was I to do next?

Some Googling later and ImageVoodoo was my knight in shining armor! ImageVoodoo is a JRuby gem which has like for like API methods for ImageScience (an image processor supported by attachment_fu). Everything looked like the gallery could finally begin working again. However the next stumbling block was going to appear, ImageVoodoo doesn’t support image sharpening. I was wondering at this point just how much more annoying this could get; this project was only meant to be something quick to get me back into writing Ruby again, but was quickly turning into a complete headache. After some more eventual playing I again had to admit defeat, not so much through not wanting this to work or lack of knowledge, but the want of a working gallery was greater at this point.

Where am I at now then? Well, the gallery is operational, but image sharpening & image compression are out at the moment. So thumbnails look like a 4 year old has been let loose with Photoshop and somehow managed to find the resize controls but on doing so, decided to lower the quality to about 10. Meaning that my lovely photos don’t look as good as they could at this moment. However with attachment_fu storing the original image all is not lost just yet, as when I finally figure it out I can simply regenerate the thumbnails.. (well, I hope I can, haven’t actually looked into how to do that yet).

It baffles me however that RMagick, which is quite frankly a huge part of Ruby image manipulation, doesn’t have a JRuby gem. Admittedly there is RMagick4J, but it’s lacking the same problems as ImageVoodoo, no sharpening support or image compression controls. If only RMagick worked on JRuby without having to think about it…

If you’ve ever worked with Firebug you’ll know its possibly the best tool out there for aiding in web development within Firefox; thankfully there are some really handy extensions for it to make it even more indispensable!

Firecookie

Allows the same controls that you get with HTML to apply to cookies, so far I had been using the Web Developer toolbar to view/change cookies, however this allows for much easier editing. It also has a feature to show you when cookies are changed, which is great for debugging.

YSlow

This is useful to work out either why your site is running slow, or for you to make improvements to make it faster. It has various rule sets which can be found here which define what exceptions it allows.

It’s going to be that time of year again soon, when a browser gets updated and all developers frantically have to make their websites compatible and all working fantastically. However, this time around the team behind Internet Explorer 8 are going to make this much easier on all us devs.

They are doing this by allowing the web-site itself to control which rendering engine IE8 will use when you browse. This will mean if you site is already compatible with IE7, it can also be compatible with IE8 by simply including one line of HTML in the header…

<meta http-equiv=”X-UA-Compatible” content=”IE=7″ />

This will apparently force newer version of IE8 to render the current page in IE7’s format, which from a standpoint on the priority of getting things upgraded, can allow it to go a few notches down on the list rather than being the number one.

For once it will be nice to worry about feature sets, improvements and bug fixing rather than worrying if the styling will still work when the next version of IE comes out.

Thank you Microsoft IE team!

This is sometihng that baffled me for a good few months until I finally figured out what the hell was going on. If you develop in rails chances are you’ll either be using Mongrel or WEBrick to run it locally, but sometimes when I went to start either of them I would get a “This port is already in use”, which wasn’t exactly fantastic.

I would look around to try and find out what was using it, but “netstat -a” doesn’t help much, it just says its taken by a local device on the machine, although it wasn’t just port 3000, it’s the entire range of 3000 to 3030 were “in use”.

Some days it would work, some days it would fail, from what I could tell it was completely random whether I would be able to start up the server. After senses of humours were lost to the pixies I finally worked it out, and it comes down to a little green icon with two arrows…

It’s ActiveSync, and you’ll have this program running if you have a PocketPC phone and if the phone is connected, say goodbye to those port numbers. It steals 3000-3030 straight away when you plug it in. Some solutions I have found are to unplug my phone, start Mongrel, then plug the phone back in. Although again, this working is random and highly frustrating. Currently I have resorted to unplugging my phone when I am developing in rails and plug it back in when I’m done.

I actually figured this out sometime last year, but saw someone come into #rubyonrails having the same problem and I thought I should probably share it.

This was more of a test to see how easy it would be to scrape some data using Ruby as I usually use PHP to do this for me. However when using PHP it always seems to be a mess of explodes and regexp’s to get what I want. So I wanted to see how other languages do it.

So first things first I found out which packages/libraries were around for Ruby, and admittedly there’s quite a few, however upon first look what appear to be the better ones don’t work out of the box. Which is a big shame, as I spent a good few hours trying to get scRUBYt! working on my Windows XP development machine, but in the end I had to admit defeat. It just wasn’t going to work properly even though I had followed every guide you could find to get it working in a Windows environment. I was a bit disheartened as this appeared to be the best one out there as its first example was exactly what I was looking for.

Alas, I moved on, had a quick look at Hpricot, but it didn’t seem to do what I wanted easily. After trying to find some examples of how they all worked I found srcAPI, which was very similar to scRUBYt however no matter how much I tried, I could only find one example, which was its own eBay one. It had a nice installation, just a gem, which worked first time (which one usually expects), from there I tried their example. I had read on their website that it uses Tidy to do the HTML cleaning, but you could use it without that if you told it to use the built in one, however it warns you that this should only really be used for testing, and and that was what I was doing I told it exactly that.

Scraper::Base.parser :html_parser

To cut to the chase, nearly 3hours, much googling and frustration later and the simplest of process’s it was refusing to scrape part of a table I already had. Telling it to use the built in one instead of Tidy was my downfall, the second I switched it back to Tidy (and a quick re-ordering of the loading code to stop it trying to load the linux version first) and it was working perfectly. This was someones comment about it all…

Phases of scrAPI usage:

1. Elation – Wow, this is so easy and powerful. I’m gonna scrape the world!!!!!

2. Despair – What the hell is the syntax for the selectors, I’m so confused, and there are no docs

3. Elation – scrAPI has great test coverage, you can learn everything you need to know about the selectors from the tests.

Couldn’t agree more, although instead of getting help from the test cases I got help within a scrAPI cheat sheet I found. Without that I would have probably given up and tried the next one in line.

So I’m now testing and doing more complex scraping with srcAPI, but for anyone who is a beginner to Ruby you may find this task quite challenging and probably beyond your scope of knowledge to begin with. The lack of documentation and examples for this made it increasingly hard, and most was done with guess work on how it was interpreting the HTML source code.

Bottom line, if you can get scRUBYt! working, go with that, it’s very powerful and from the examples I have seen, very programmer friendly and will allow you to get to the data you want fast. But if you can’t get it on the go, then there’s many others available including many I haven’t mentioned, but Google knows all! (apart from examples of srcAPI )

Ever find that you are searching through various programming language’s API sites, and wished that there was an easier way of doing it all? Well I found this site the other day called GotAPI, and on their site it allows you to specify which API’s you are interested in, and from there you get a very clean interface in which to search through them with.

So instead of having to have 5-10 bookmarks for all the different languages I now only need the one. It’s very fast searching and uses the standard web 2.0 style of ajax. But it’s done in a way which is very useful rather than showing off the fact its ajax and it being intrusive and annoying.

You can have as many languages as the site supports open as they all load up in tabs at the top, so you can search between them by just clicking the correct language. It also includes API’s for various databases and major sites such as Flickr as well, so it’s not just useful for programming.

The bottom line, there’s no adverts, it’s fast, free, and all in one place. GotAPI