md5 in Node.js

If you come from PHP background, you’re used to PHP’s global top-level functions for everything. Some people say it’s PHP’s curse, others praise it. I’m doing some Node.js stuff lately and needed equivalent of PHP’s md5sum() function. Turns out, it’s really simple and included into base Node.js install. You need to use the “crypto” module and generate md5 hash using createHash. “createHash” might sound confusing as data is not really hashed by the function. You create hash and then add data to it. After all data is in, you read the digest:

var crypto = require('crypto'); 
crypto.createHash('md5').update(data).digest("hex");

That’s all folks. Happy noding :)

Node.js and Express serving the same content for HTTP and HTTPS via SSL

In my previous post I explained how to set up SSL for Node.js/Express. Now, I want to serve the same content using the same logic for both http and https, and I don’t want to duplicate my code.

The idea is to move everything involving Express app. into a function. Call the function for both http and https server. If you have global variables, make sure they are outside of this function:

var apps = express.createServer({key: pkey, cert: cert, ca: [dad1,dad2]});
var app = express.createServer();
apps.listen(443);
app.listen(80);
startServer(app, false);
startServer(apps, true);

function startServer(app, isSSL) {
    app.configure(function () { // just some sample code
        app.use(express.cookieParser());
        app.use(express.bodyParser());
        app.use(connect.static(‘public’));
    });

    app.get(‘/’, function(req, res){
        if (isSSL)
            res.end(‘Hello HTTPS’);
        else
            res.end(‘Hello HTTP’);
    });
}

This works, although I hope there is some nicer solution.

Setting up real SSL with Node.js and Express

I got my single-domain certificate from Godaddy. Suddenly, I got myself with .key file, .csr file, and two .crt files. Most examples you can google on the Internet use self-signed certificates (which is basically useless for Internet use) and .pem files. Wft is .pem, you might ask?

After wasting hours trying to get this to work, I finally did. I hope more posts like mine get written and reach google index, so that people trying to set up production systems don’t have to waste time. Here’s how I did everything, step by step:

1. create your private key and certificate-request file. I used the command suggested by Godaddy as it requires 2048 bit key. Suppose your domain is domain.com:

openssl req -new -newkey rsa:2048 -nodes -keyout domain.key -out domain.csr

Most of the questions that follow are straightforward. Godaddy suggest you use domain.com for “Common name” field.

2. log into Godaddy, go to SSL menu and select Manage option. You actually buy a credit for SSL cert. so you need to “use” it, and then request a certificate. After using the credit and pressing “Launch” button I was welcomed with a screen saying zero (0) in all categories (certificates, requests, credits, etc.). This was rather confusing. Googling around, I found the solution: go to credits or certificates even though it says zero. After the page loads, an option appears to “update” the list. Click this and your credit shows up finally. Now, you can “request” the real certificate. Paste the content of domain.csr file you created in step 1. and wait for GD to create the cert.

3. after the cert is created, download it (there’s a download option on the certificate screen). You’ll get a .zip file containing two .crt files: domain.com.crt and gd_bundle.crt. First file is your SSL cert. The second file contains CA certs. of Godaddy that were used to digitally sign you cert. gd_bundle.crt might contain multiple certs. of which most browsers only need the first one, but it’s better to install both. I’ve read some reports that some clients (ex. Android) require both to be installed properly.

4. Time to add all this to our Node.js/Express setup. It’s a little bit different if you don’t use Express (you need to call http.setSecure() with credentials):

var express = require('express');
var privateKey = fs.readFileSync('domain.key').toString();
var certificate = fs.readFileSync('domain.com.crt').toString();
var dad = fs.readFileSync('gd_bundle.crt').toString();
var app = express.createServer({key: privateKey, cert: certificate, ca: dad});
app.listen(443);
app.get('/', function(req, res){
    res.end('Hello SSL');
});

Supplying “ca” field to createServer is crucial, and missing from most examples on the net since they use self-signed certs.

Now, open http://domain.com and you should see the welcome message.

5. This works fine for my Firefox test. However, if you inspect the certs, you’ll see that only one CA cert. is sent. To send both, we need to split gd_bundle.crt into two files and tell express to read both:

var dad1 = fs.readFileSync('gd_bundle.crt').toString();
var dad2 = fs.readFileSync('gd_bundle.crt').toString();
var app = express.createServer({key: privateKey, cert: certificate, ca: [dad1, dad2] });

That’s all. I hope this saved you some time. In case it did, please follow me on twitter @mbabuskov, as I will post more Node.js stuff as I develop my applications.

Why PHP is better than JavaScript

I started developing a small project using node.js with express and socket.io. Node is a nice server and socket.io is great. However, I’m having issues with javascript. Currently, two things really get on my nerves:

1. the plus operator

Most of the hard-to-debug bugs in my javascript code come from the + operator. It decides to concatenate strings instead of add numbers. Considering that all stuff that goes over the wire (i.e. socket.io) is treated as strings, it’s really painful and ugly to have parseInt(…, 10) everywhere. PHP solves this issue with simple dot operator. Simple, no-brainer and always does what you expect. You don’t have to think where does the data come from.

2. foreach

I miss PHP’s foreach so bad. Consider:

for (ix in really.long[expression].toGet.theStuff) {
    if (really.long[expression].toGet.theStuff[ix].value < 10 && really.long[expression].toGet.theStuff[ix] > 5) {
        ...do something

versus PHP’s:

foreach ($really.long[expression].toGet.theStuff as $ix=>$value) {
    if ($value < 10 && $value > 5) {
        ....do something

Of course, one could assign the array element to some local variable, and so I have local variables all around wasting code lines and making code error prone (if you need to change the collection you are iterating, you have to change in two places).

Google multi-account get screwd up again

Looks like guys at Google really have trouble with multi-accounts and sessions. Everything was working fine for months now, but they messed it up again. What does the problem look like:

  • I have 2 google accounts, one @gmail.com and other @mydomain.com
  • I cannot login into @gmail one directly. I have to log into @mydomain and then use the “switch account” feature
  • I cannot bookmark both gmails. Although bookmarks are different, both open @mydomain account
  • Most other google services I use (ex. Analytics) are tied to my @gmail account. I cannot access those at all, unless I log out of everything, clear all the cookies and then log just into @gmail.

My user experience with google is getting worse every day:

  • multi-account login problems
  • google docs become painfully slow when spreadsheets grow 300+ rows (only about 10 columns though)
  • search is polluted with g+ spam, translation offerings, etc.
  • on one of my computers it shows my location in the wrong country and I cannot find a way to tell it differently.

If someone build free replacements, I would surely give those a shot.

GMail ignores Reply-to header [SOLVED]

I have a conact form on my website. People can leave their e-mail so that I contact them back. I set it up so that e-mail is sent From my e-mail address (general rule: never put user’s e-mail in From field), and Reply-to set to user’s e-mail address.

However, when I click “Reply” in GMail, the reply gets sent to back to me. Looks like some glitch in GMail’s design, and they did not bother to fix it for a long time.

The solution is rather simple, just change the From field to some other address you own (different from GMail account address). For example if your e-mail is office@example.com, you can use support@example.com in the header. After this little change, Reply-to started working properly.

I hope this helps someone.

How to extract mp3 from YouTube video using Linux?

It’s rather easy. I used 3 components:

  • DownloadHelper extension for Firefox
  • MPlayer
  • lame

When DownloadHelper is installed and you open a YouTube video, it gives you the options to download the .flv or .mp4 file to your computer.

After the file is downloaded, we can use MPlayer to play it and also to dump the audio. It’s nice because it plays both .flv and .mp4, so you just need one program. To dump the audio to .WAV format use:

mplayer -ao pcm:waveheader FILENAME.flv

This will create file called audiodump.wav. Now, use mp3lame to encode it to mp3 format. You can also use oggenc to convert it to OGG if you perfer open formats.

lame audiodump.wav song.mp3

That’s all. I put these commands in a simple shell script (video2mp3.sh):

mplayer -ao pcm:waveheader $1
lame audiodump.wav $1.mp3

Run it from command line like this:

. video2mp3.sh FILENAME.flv
Lean Startup vs Rework
Some time ago I finished reading the Lean Startup book by Eric Ries. Although I have been using some techniques from it before I have learned a lot. Yesterday, I found a mention of Rework on some website. Rework is a book by founders of 37 Signals, which I also read before. I started to compare the content of Lean Startup and Rework and I got some interesting conclusions.
Rework and 37 signals business model is really only a sub-set of Lean Startup philosophy. Basically, Rework stops somewhere during lean startup process and says &#8220;we&#8217;re content with this&#8221;. You build MVP, test it with customers, tweak a little bit and whoa, if you get good product market fit, what&#8217;s next? Depends who you ask: 37signals guys would tell you: &#8220;Well done, now enjoy your success&#8221;. They refuse to grow business, add features and go for a larger market. I&#8217;m not saying this is bad, sometimes you need to know what is your field of competence and stay there.
Which one would you follow? It depends on your personality. If you&#8217;re going to become a serial entrepreneur, rework is not enough. If you have an urge to move forward, discover new horizons, you might need to use Lean Startup in each new project again. After all, one can hardly call 37signals a startup anymore. They behave like established business, not a startup. As DHH said on Twist, &#8220;if you&#8217;re not doing your best idea now, you&#8217;re doing it wrong&#8221;. But, how do you know what is your best idea going to be if you do not explore? Maybe something looks like my best idea now, and I should be working on it. But, by the time that project becomes mature and stable, I might get a dozen of better ideas. And once I can turn the reigns of the current project to some good manager, I can go back to &#8220;startup&#8221; mode and explore new boundaries.

Lean Startup vs Rework

Some time ago I finished reading the Lean Startup book by Eric Ries. Although I have been using some techniques from it before I have learned a lot. Yesterday, I found a mention of Rework on some website. Rework is a book by founders of 37 Signals, which I also read before. I started to compare the content of Lean Startup and Rework and I got some interesting conclusions.

Rework and 37 signals business model is really only a sub-set of Lean Startup philosophy. Basically, Rework stops somewhere during lean startup process and says “we’re content with this”. You build MVP, test it with customers, tweak a little bit and whoa, if you get good product market fit, what’s next? Depends who you ask: 37signals guys would tell you: “Well done, now enjoy your success”. They refuse to grow business, add features and go for a larger market. I’m not saying this is bad, sometimes you need to know what is your field of competence and stay there.

Which one would you follow? It depends on your personality. If you’re going to become a serial entrepreneur, rework is not enough. If you have an urge to move forward, discover new horizons, you might need to use Lean Startup in each new project again. After all, one can hardly call 37signals a startup anymore. They behave like established business, not a startup. As DHH said on Twist, “if you’re not doing your best idea now, you’re doing it wrong”. But, how do you know what is your best idea going to be if you do not explore? Maybe something looks like my best idea now, and I should be working on it. But, by the time that project becomes mature and stable, I might get a dozen of better ideas. And once I can turn the reigns of the current project to some good manager, I can go back to “startup” mode and explore new boundaries.

Twitter Timestamp out of bounds [solved]

Some time ago, automatic sending of status updates from one of my applications stopped working. I haven’t changed anything on the server, so this was strange. Looking into error message from Twitter:

[request] => /1/statuses/update_with_media.json 
[error] => Timestamp out of bounds

well, that’s strange. I recall time was moved to DST (daylight savings) Summer time in US recently, and apparently so did Twitter’s servers. The rest of the world - bah, they don’t seem to be interested, they can barely run the servers for US users apparently.

So, until DST change comes to the rest of the world, we need a hack. At first I thought that Twitter does not like timestamps to be in the future, so I thought about decreasing the timestamp of tweets. That did not work. So, I tried to increase the timestamp and everything is working now. The change is quite simple, just increase OAuth oauth_timestamp by a couple of hours and you’re done. In PHP OAuth client it looks like this (I added 5 hours):

  private static function generate_timestamp() {
    return time()+5*3600;
  }

What’s the conversion on e-mail newsletter with a call to action?

I have measured that in the past week. One of my websites has more than 500.000 user accounts. I picked the users who were not on the site in the last week, because they have already seen the news on the site, and got some 384.000+ distinct e-mail addresses. I needed to contact them regarding an important issue about the website. The e-mail was composed like this:

Hello,

you are reading this mail because you are a member of [mysite link].

Issue explained and link with [call to action]

Regards,

Your webmaster
[mysite link]

Links where not plain text, but special URLs I used to track the clicks.

I sent the e-mail slowely over a 7 day period. I wanted to track weekends and working days as well.

And here are the stats:

  • 384408 different e-mail addresses
  • 13966 bounced back (3.6% bounce rate)
  • 9345 clicks (yielding 2.5% conversion rate)

Hot spots in the e-mail message:

  • 23% clicks on link in the first sentence
  • 65% clicks on call-to-action
  • 12% clicks on website link in signature

Here are the weekday stats:

  • Monday 13%
  • Tuesday 16%
  • Wednesday 18%
  • Thursday 18%
  • Friday 15%
  • Saturday 9%
  • Sunday 11%

twitter.com/mbabuskov

view archive



Ask me anything

Submit