When I discover delicious it was too late for me since at the time I had too many bookmarks already tagged in Yahoo’s Myweb. But since then I’ve always wanted to move all my bookmarks to delicious but didn’t found how to do it, until now.
Some days ago I found this python script that makes use of myweb and delicious web services to migrate the bookmarks.
Unfortunately the script wasn’t very reliable, it was crashing quite often and, since I have more than 1500 bookmarks, it was quite painful since every time I launched it, it was starting from scratch.
So I decided to build a new script (actually two)… in Ruby.
Step 1 : prerequisites
First of all you must install Ruby and rubygems on your system.
Then you must install rubilicious gem.
Step 2 : export myweb bookmarks
Here’s the first ruby script that you must run:
require 'uri'
require 'rexml/document'
# YOU WANT TO MODIFY THIS
yahoo_id = 'pix'
myweb_appid = 'YahooDemo'
myweb_urlsearch = 'http://search.yahooapis.com/MyWebService/V1/urlSearch'
request_start_idx = 1
request_max = 50
while(true)
url = myweb_urlsearch + '?' + {
:appid => myweb_appid,
:yahooid => yahoo_id,
:results => request_max,
:start => request_start_idx
}.to_a.collect {|kv| kv[0].to_s + '=' + kv[1].to_s}.join('&')
retries = 3
while(true)
$stderr.puts(" - Requesting URLs from [#{request_start_idx}] (#{retries} retries left) -> #{url}\n")
http_resp = Net::HTTP.get_response(URI.parse(url))
if(http_resp.code.to_i < 200 || http_resp.code.to_i > 299)
$stderr.puts(" ! request failed [#{http_resp.code}]\n")
retries -= 1
sleep(5) && next if(retries > 0)
$stderr.puts(" ! too many retries, something is broken!\n")
exit
end
break # exit while loop
end
xml_data = http_resp.body
# extract event information
doc = REXML::Document.new(xml_data)
tot_results = doc.root.attributes['totalResultsAvailable'].to_i
break unless tot_results > 0
doc.elements.each('ResultSet/Result') do |r|
puts [
r.elements['Title'].text,
r.elements['Summary'].text,
r.elements['Url'].text,
r.elements['Tags'].collect() {|e| e.text.to_s}.join(','),
].collect {|v| URI.escape(v.to_s) }.join('&') + "\n"
request_start_idx += 1
end
end
Copy this code and paste it to a file (like myweb_export.rb). You also want to set the yahoo_id variable with your myweb ID.
Then you can run the script:
Step 3 : import bookmarks into delicious
For the final step you’ll need the script below:
require 'uri'
r = Rubilicious.new('USER', 'PASSWORD')
$stdin.each_line do |l|
title, description, url, keys = l.split('&').collect { |i| URI.decode(i) }
keys = keys.split(',').collect { |k| k.strip.gsub(/\s+/, '-') }.join(' ')
puts title + "\n" + url + "\n" + keys + "\n---\n"
r.add(url, title, description, keys)
sleep(1)
end
You want to set the proper values for USER and PASSWORD of your delicious account and copy the code to deli_import.rb.
The script waits 1 second after each URL since this is required by the delicious API.
Limitations: unfortunately since the Myweb web services only allow the search of public bookmarks, you’ll not be able to migrate private bookmarks. You must do this by hand or wait that Yahoo will release a Myweb webservice that supports authentication.
I’m selling some of my technical book at very cheap prices, take a look and send me an email if you’re interested.
While looking around the Facebook developers docs I’ve found a page called Programming Puzzles. Basicaly if you want to work for Facebook you better try to solve some of these puzzles with one or more of the suggested languages and send the source code along with your resume.
Next week after my university final exams I’ll try to solve some of them in Erlang and Ruby, in the meantime have a look and try yourself. 
For a university exam (Human Computer Interaction) I’m developing a realtime shared map that you can use to plan a travel with your friends in realtime while you chat with them.
The user interface
The map will be interactive and it will have some shared widgets that will provide the user with:
- user roster (who’s online)
- chat window
- shared landmarks that users can put and move on the map
- shared post-it notes associated with landmarks
- flickr widget that can be moved across the page and will show the pictures related to that place
- a shared wiki widget where you can put notes, drag landmarks to store interesting places, drag pictures from the flickr widget
All these widgets are shared across the user, their position and their content gets replicated on the browsers in realtime and every user can interact and change the information.
The architecture
The application logic is composed of two entities: the map application and the agent application.
The map application
The map application will run completely on the user’s browser, his responsibilities are:
- Manage the interface (the map and the widgets) and the interaction with the user
- Forward the interaction events to the agent application
- Update the interface based on the incoming events from the agent application (i.e. when other users interact with the map)
The agent application
The agent application can run anywhere as long at it can have an XMPP link, his responsibilities are:
- Keep a consistent state of the map and the widgets (map position, zoom level, landmarks positions, wiki data, etc…)
- Update the map status based on the messages coming from the users
- Forward the map updates to the users
Technologies involved
Frontend
The user interface is created from HTML and javascript is used to manage the page updates and the interaction events.
A google map is used as the main page widget and other small windows are created on the map and can be dragged over the page by the user.
Messaging infrastructure
The comunication link between the map application and the agent application is created by using the XMPP4MOZ extension for Firefox.
The map application encapsulates JSON objects describing map events into XMPP messages that are sent to the agent application in realtime.
The map application also receives JSON objects coming from the agent application describing map updates.
Any XMPP (Jabber) account can be used for users and the agent. I used Google Mail accounts that can be used with the very fast and reliable Google Talk XMPP server.
Backend
The backend application (agent application) is made in Ruby using the XMPP4R to manage the XMPP communication link and the Ruby JSON library to encode and decode JSON objects.
The backend will keep the status of the map and the widgets so everytime a new user joins the frontend can show the current status of the map.
The prototype
I was able to hack a quick prototype of the frontend and the backend on Friday afternoon.
Everything is working fine so far.
I was able to run the agent application and two users on the map application each on one different laptop (yeah I have three laptops… actually four :D) and watching map moving in realtime from one laptop to the other.
I also had time to create a simple widget that enables a user to put landmarks on the map that can be dragged around by any user. The agent keeps the position of the map and of each landmark so when a new user logs in he will see the landmarks created by other users and he will be able to interact with them.
What’s next
I want to refactor a bit the frontend code to separate the core of the application that manages the messaging with the backend with the widgets that let the user interact with the map. Ideally you should be able to plug functionality into the core application by including self contained modules.
I’ll plan to share some of the code once I feel it’s kinda useful…
