Ruby Code to Scrape Images from TwitPic URLs

Twitpic logo

I’ve been trying to find an easy way to re-grab my TwitPic images back off TwitPic to be re-syndicated on a communal family website I am trying to create.  I found a great site which had some Rails code that scraped the image, so I took the code and modified it to make it compatible for pure Ruby (too be honest it didn’t need much modifying at all) .

Here’s my modified snippet of code that I’ve been using to grab the image from Twitpic with the Hpricot gem:

require 'rubygems'
require 'net/http'
require 'hpricot'

def rip_twitpic(url)
  begin
    code=url.match(/[w]+$/).to_s
    unless code.empty?
      uri=URI.parse(url)
      resp=Net::HTTP.get_response(uri)
      html=Hpricot(resp.body)
      html.at("#photo-display")['src']
    end
  rescue Exception => e
    puts "Error extracting twitpic: #{e}"
    url
  end
end

Just as it appears, this method will return the URL of the image embedded on the page that the TwitPic URL points too.

This will form the core part of a little project which will allow you to scrape TwitPic images and send them to a server of your choosing. I’ll try to release this ASAP but in the meantime, I thought a few of you might find this useful. I know I do.

Getting SSI to work in MAMP

It’s no secret that I adore my Mac, mostly because it just makes my life easier.  I enjoy the security of OSX, I love it’s responsiveness and I love that my operating system doesn’t punish me for installing applications and development stacks just to ‘play with them’.  A while ago, while looking for a stack that would let me quickly and effortlessly get Apache, MySQL and PHP5 working together without having to muss about with configs and the such, I discovered MAMP and frankly, I adored it instantly.  After a rather large download, it was effortless to install and running it was a cinch.  And better still, the server applications only run when you open the MAMP application and tell them to run, so that you dont loose vital system resources running services you dont need.

Additionally, my IDE of choice, NetBeans can integrate directly with MAMP, so that when you create (for sake of argument) a new PHP project and run it, it automatically puts the project files into the MAMP Apache directory (to be honest you do need to configure the path when you create the project, but its no big deal) making development painless and convenient.

But I did hit a snag recently when for a university assignment I was required to do a XHTML website using HTML Server Side Includes.  I was shocked when I ran my website and my SSI didnt work.

But there is a solution.

To make .shtml work, I deleted the comment-symbols ( # ) in the file http.conf (find it in /Applications/MAMP/conf/apache/ and at the time of writing was near lines 982:

# To parse .shtml files for server-side includes (SSI):
# (You will also need to add "Includes" to the "Options" directive.)
#
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml

High Performance & Multi-threaded SCP Using RSYNC

Recently, I had the somewhat laborious job to backup a stack of websites and blogs from my remote web server.  Initially I tried do it with a simple SCP command; but after letting it run for about an hour, it was obvious that it was just too slow and taking too long downloading each file one at a time.

After talking complaining to a friend he suggested using the RSYNC command. Surprisingly it was incredibly very easy to get it working, you simply issue:

 rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/

Obviously changing the appropriate parts for your case.

I found it to be at least 10 fold faster (or more) than SCP on it’s own, and better still, RSYNC will resume when SCP wont!  Try it and see for yourself.

Project Euler Problem 52 Solution

It can be seen that the number, 125874, and its double, 251748, contain exactly the same digits, but in a different order.

Find the smallest positive integer, x, such that 2x, 3x, 4x, 5x, and 6x, contain the same digits.

My solution in Ruby:

i = 1;
answer = nil;

while answer == nil
  i += 1
  x = i * 2
  base = Array.new(x.to_s.scan(/./))
  matchbase = true
  for t in 3..6
    break if matchbase == false
    x = i * t
    digits = Array.new(x.to_s.scan(/./))
    digits.each { |d| matchbase = false if base.index(d) == nil }
  end
  answer = i if matchbase == true
end
puts "Answer: #{answer}"

Left4Dead and Left4Dead 2 Available on OSX in October

Back in March, the Mac gaming world got excited when Valve announced their Steam gaming software was coming to the Mac — along with Left 4 Dead 2, Team Fortress 2, Counter-Strike, Portal, and the Half Life series.  I was shocked at just how quickly the Valve catalog was being ported to OSX, but then, the announcements stopped as suddenly as they started; alegedly sue to a number of graphics and OpenGL bugs issues that Valve helped Apple sort out.  Today, I found this little gem:

We’d previously heard tell that now that those graphic issues are fixed, Valve as hard at work to bring Left 4 Dead and Left 4 Dead 2 to OS X by October… and now, if a casual mention over at Macworld is anything to go by, it looks like that date might have been further locked down to October 5th, along with the latest Left 4 Dead and Left 4 Dead 2 add-on pack, “The Sacrifice.”

So only a few more sleeps until all Mac users can help keep the hordes of zombies at bay with their Windows buddies.

Do I Look Like I’ve Lost Weight?

I’ve been on a new diet recently, and it’s had really good results.  I started out 123.8Kg and a little over a month later, as of today, I’m down to 111.9Kg.

Now obviously by the numbers, this is a great result, but the sad fact is that I dont feel different.  I dont feel like I  look different either, despite what friends and family have  said.  I’m wondering if being obese (that word cuts like a knife, doesn’t it?) effects you psychologically more than we think.  This then, after loosing 10cm off my stomach circumference, which should be clearly obvious – just isn’t to  me in the mirror.  I wonder if I might have some kind of body image problem, or worse still, a kind of eating disorder?

I like to think that I am a level-headed person, but over the past 3 or 4 years I’ve been becoming increasingly self-conscious about my body – to the point that I no longer go swimming!  I’m not really sure what to do from here, but since this diet has been working, I guess I’ll just keep doing it, and hopefully, when my current clothes simply do not fit any longer, I’ll feel differently.

Monstrous Discrepancies

I saw this image late last night and instantly loved it.  Its so true how far our perceptions and feelings can be from actual reality.  I’d like the original artist to make it into a poster so I can put it up somewhere where my children will always be able to see it.

It’s so great in fact, that I’ve mirrored it here should the original URL suddenly become no longer be available.

Windows 7 Drivers for Apple’s USB Ethernet Adapter

Listen up, all you Boot Camp’in Mac users … Can’t find Windows drivers for the Apple USB Ethernet adapter? Want to use use the adapter on another Windows machine? Are you crazy?  Well, it turns out it CAN be done, with a little bit of trickery.

There is a lot to be said for Apple’s simplicity of design.  Even their adapters and cables look as if they were pain stakingly and lovingly hand crafted by an eccentric, gay, Swedish man. Everything just looks better.

Recently, after the onboard NIC died in my Acer Aspire L3600 (which runs Windows 7 x64, and I use as a dedicated Windows Media Centre).  After flashing the BIOS and jumping through several hoops with no avail, I needed to go looking for another way to get a wired >=100Mbit/s network interface into the machine.  Since its ultra-compact form factor makes an internal PCI option impossible, I needed to go looking for an external (USB) option.  It didn’t take long to realise that my options were going to be extremely limited and after checking out my nearby computer retailers, I had only two options.  A reasonably generic SWANN adapter, or Apple’s USB Ethernet Adapter – both were in stock at my local Dick Smith Powerhouse.  The problem with the SWANN adapter, is that its a single solid block, and since my Media Centre lives very close to the wall, it wasn’t desirable, while The Apple adapter can be flexed to a right angle.

Problem is, the Apple adapter says (on the back) that it can ONLY be used with an Apple MacBook.  But the adapter’s drivers ARE included as part of BootCamp, which means it can function when running Windows on a MacBook. So with a little trickery, you can get it working on any old Windows 7 or Vista PC (32 or 64bit editions).

  1. First, you need to download the driver for the adapter, either the 32bit or 64bit version.
  2. Next, extract the zip file and locate the file Ax88722.inf.  It needs to be altered in order to get the device drivers to be installed.  In order to simplify the process, I’ve simply got the tweaked version here, for you to download. Just replace the original Ax88722.inf file with this one inside this zip file.
  3. Next, attach your USB ethernet if you have not done so. Launch device manager (right-click on “computer” and select “Manage”). Locate the lonely unknown device “Apple USB Ethernet” and right-click it to select “Update Driver Software”.
  4. Select “Browse my computer for driver software” and in the file browser dialog select the folder of your recently modified .INI file and continue the wizard. This should bring your Apple USB ethernet to life!

Apparently there are drivers for 32bit versions of Windows XP, put together by the BootCamp community, if you’re an XP user and feeling lucky you can try your luck with this link (but like the rest of this post, use it at your own risk).

Download Microsoft .NET Framework 3.5 SP1 Standalone Full Redistributable Setup Installer

Many applications uses Microsoft .NET Framework 3.5 as development platform, and thus requires .NET Framework to be installed beforehand, else the installation will request to download and install .NET Framework from Internet. On offline system without Internet access or online server with slow downloading speed, the requirement to download setup files through web may hit the wall – a no go.

kamagra shipping

Microsoft initially just provides a minimal size dotnetfx35setup.exe download which is a bootstrapper that will still need to download more files via Internet. For users who prefer to perform offline installation or install .NET Framework 3.5 SP1 without waiting for download to complete will have to download and save a copy of full complete standalone or redistributable Microsoft .NET Framework 3.5 SP1 setup installer, which is finally published by Microsoft.

Microsoft .NET Framework 3.5 SP1 (Service Pack 1) is a full cumulative update that contains many new features building incrementally upon .NET Framework 2.0, 3.0, 3.5, and includes cumulative servicing updates to the .NET Framework 2.0 and .NET Framework 3.0 subcomponents. See KB951847 for list of changes and fixed issues in the .NET Framework 3.5 Service Pack 1.

Download full package of Microsoft .NET Framework 3.5 SP1: dotnetfx35.exe (231 MB)

For known issues and release notes, refer to Microsoft .NET Framework 3.5 SP1 Readme.

Native UML Module for NetBeans

UPDATE: There is a more recent post regarding the Native UML module for NetBeans here.

I have been a long time user of NetBeans IDE. I really like the simplicity and how it is almost an all-purpose IDE for virtually any language you can think of.

However, I was horrified recently when I upgraded to version 6.8, to discover that the UML module was gone, and instead NetBeans insisting I use a third-party UML generator.  So I tried it, and hated it immediately.  I considered rolling back to an earlier version, but first I thought I would try a little voodoo.

I noticed that version 6.9 was released and so I downloaded it, and installed as per normal.  I then found this page (the 6.9 release candidate zip download page) which listed a file called ‘netbeans-6.9-201005312001-ml-uml.zip’.  Ah ha.  Since I am on OSX, I opened the package contents up from Applications, unzipped the uml.zip file and dropped the new UML directory into the installation directory.  I started up NetBeans and voila:

I tested the new project, and it worked like a treat (some people who tried various other hacks reported that even though the project appeared, there was issues saving/opening them).

I hope this works for you too, and of course make sure you backup any existing projects – dont come crying to me if you destroy your work trying to implement this!