WTF? Pagestackr was down!

Not anymore (I hope).

Last Sunday, we experienced more than one type of failure that caused Pagestackr to be down for several hours, intermittently.

Firstly, a friend said to us that our signup wasn't working. That's odd, since we are using exception notifier to send us any exception by email. Checking the server's log and we found out what happened: our external SMTP server was denying access from our server.

We use a SMTP server at railsplayground.net. We were using it in some project's development and we thought it would be a goood idea to use it with Pagestackr.

Well. We hurry to railsplayground support and the response was that their SMTP server was OK. "Check your app", they said. But it was working before. And we didn't change any line of code (due to the Rails Rumble restriction).

Oddly, after a few hours of agony, the SMTP connection resume working and has been working since then.

The second failure has to do with our del.icio.us import. A bad implementation on our side when treating del.icio.us error messages caused mongrel to lock in this thread. Combining that with an almost-infinite loop, and now we can lock ALL our mongrel instances. Nice, ahn?

This bug is very simple to solve (and we did have a solution for this), but, of course, we are not allowed to make changes on our code.

The solution? We used a iptables rule to limit access to del.icio.us' api to one request per 500 seconds:

echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -I OUTPUT -p tcp -d api.del.icio.us -m state --state NEW -m recent --set
iptables -I OUTPUT -p tcp -d api.del.icio.us -m state --state NEW -m recent --update --seconds 500 -j REJECT

(actually, we used api.del.icio.us' IP address. We don't want to resolve the hostname with packet filtering.)

This way, when our loop sends a new request that would lock one mongrel, it's instantly denied by iptables and our mongrel can keep serving other requests.

So, our del.icio.us import IS working, but in a time-limited way. (yeah, it sucks... but we have to do that to keep Pagestackr running).


Finally, I want to make a clarification on voting. We are a bunch of college guys, and our friends heard about pagestackr and it spread out in the entire campus. We have emailed some people too, but we explicitly said to vote as fairly as possible. And to give a look on other apps. But, Carter, feel free to remove any votes that you consider to be invalid.


We (me, Hugo, Jose and Mario) are available to talk, and we promise to be more present at #railsrumble. You can reach me at george.guimaraes@gmail.com (email or gtalk).


PS: Congratulations to irksome, linga and rubybrigade. Really loved your apps.

Pagestackr. Bookmark inteligente

No fim de semana passado, eu, Hugo, deDeus e Mario participamos do Rails Rumble 2007, uma maratona competição de programação utilizando Ruby on Rails.

A idéia é simples: produzir um aplicativo web usando Rails em 48 horas!

Foram 48 horas muito corridas, mas como o Hugo mesmo disse: "O George foi o que mais dormiu...". Então não posso reclamar muito.

E é com grande orgulho que lhes apresento o Pagestackr. Um aplicativo de bookmarks mais inteligente que os que rodam a web por aí. Experimentem. Nosso grande diferencial é que nós indexamos a página inteira do seu bookmark, permitindo que nossa ferramenta de busca seja mais precisa do que a busca por tags, por exemplo.

Se você gostou, peço que vote no Pagestackr para que nossa equipe ganhe a competição. Neste momento, já estamos em 1o. lugar! Mas ainda tem muita votação pela frente.

Para votar, entre em http://vote.railsrumble.com/teams/pagestackr/visit, cadastre-se e vote nas estrelas. E ajude uma equipe de desenvolvedores totalmente brasileira a ganhar reconhecimento! Você pode acompanhar a votação (e conhecer os outros competidores) em http://vote.railsrumble.com/.

E lembre-se. Nosso aplicativo foi feito em apenas 2 dias. Tem várias coisas que queríamos mudar e adicionar, mas o regulamento proíbe qualquer alteração posterior do código. Depois eu posto mais informações técnicas sobre a arquitetura do Pagestackr, ok?

Obrigado pelos votos!