HttpUtility.UrlEncode vs. Uri.EscapeDataString vs. UrlPathEncode

How to proper encode and decode values that were passed from Client to Server and vice versa. Client side is using Javascript methods : encodeURI( ) and encodeURIComponent ( ) ?

This seem to be a very fundamental and popular issues – the real complication is when you’re using MVC routing to pass route-data values between controllers. In our solution, we’re using a proprietary method to perform aliasing-dealiasing, which add up more complexity to perform proper value encode/decode.

I’ll update this blog with the solution I found later.

Set your password wisely, to remember them easily, for managing multiple accounts securely?

Most popular (and wrong) passwords

An average Internet citizen will have Facebook, Twitter, probably Pinterest or LinkedIn account – and then your Gmail, Yahoo mail, your Internet banking ID, your favourite forum account and the list goes on.

It turns out that most of us choose a common password for every site – what is the implication ? Well, if somebody gains leverage on one of your account.

It’s true that human brain can only process and memorize certain amount of data – that’s why people tend to choose easy password to remember. And it’s wrong, totally WRONG !

Here are the worst and unfortunately most popular 25 passwords in 2012:

#              Password                Change from 2011
1               password                 Unchanged
2               123456                    Unchanged
3               12345678                Unchanged
4               abc123                     Up 1
5               qwerty                     Down 1
6               monkey                    Unchanged
7               letmein                     Up 1
8               dragon                     Up 2
9               111111                    Up 3
10             baseball                   Up 1
11             iloveyou                   Up 2
12             trustno1                   Down 3
13             1234567                  Down 6
14             sunshine                  Up 1
15             master                      Down 1
16             123123                    Up 4
17             welcome                  New
18             shadow                    Up 1
19             ashley                      Down 3
20             football                     Up 5
21             jesus                        New
22             michael                     Up 2
23             ninja                         New
24             mustang                   New
25             password1               New

Is your password in the list ? If so, read on for my suggestions.

So, how to set password wisely ?

This suggestion below is totally based on my personal experience and I shall not bear any responsibility – OK I am done with disclaimer and if you’re still here, let’s move on:

1.  Separate your important accounts from unimportant ones.
For me, I’ll set a separate password for my junk account (forums, secondary Facebook account, Yahoo mail - sorry Yahoo, your spamming filter doesn’t work !).

For important account such as credit card, Paypal, Amazon or banking-related, I’ll pick a different and much longer password.

That way, at least the chance of getting my credit card hacked reduced by 50%.

2. Set your password long & complex enough for hackers
Every password can be crack given enough time with brute-force.  According to the calculation at this page, at brute-force speed of 1B guesses/ second (cluster servers, super computing node, or a botnet networks) :

• It’ll be matter of minutes or hours to crack if your password is just pick from alphanumeric values.
• But it’ll take 83.5 days to crack if your password include all possible characters.
  which are:
  0123456789AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYyZz <SP>!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~
  

Password	Class of Attack
Length	Combinations	Class A	Class B	Class C	Class D	Class E	Class F
2	9,216	Instant	Instant	Instant	Instant	Instant	Instant
3	884,736	88½ Secs	9 Secs	Instant	Instant	Instant	Instant
4	85 Million	2¼ Hours	14 Mins	1½ Mins	8½ Secs	Instant	Instant
5	8 Billion	9½ Days	22½ Hours	2¼ Hours	13½ Mins	1¼ Mins	8 Secs
6	782 Billion	2½ Years	90 Days	9 Days	22 Hours	2 Hours	13 Mins
7	75 Trillion	238 Years	24 Years	2½ Years	87 Days	8½ Days	20 Hours
8	7.2 Quadrillion	22,875 Years	2,287 Years	229 Years	23 Years	2¼ Years	83½ Days

83.5 days on a super computing node to crack a 8-character password ? So let’s make your password AT LEAST 8-CHARACTER LONG !

3. But how to make it memorable to you ?
”Memorable” is a very subjective term and rely heavily on your personal preferences, so I would introduce only a few techniques from Wikihow – find out which one suit you most  !

How to store my password securely

Most importantly: do NOT write it down and keep it somewhere – you probably spend more time to find where do you store that piece of paper

Instead, with the popularity of today’s smartphone, make use of it: http://www.tomsguide.com/us/pictures-story/662-3-best-mobile-password-managers.html

My personal favourite is LastPass – I only wish that them implement finger-print authentication so that I can use it safer on iPhone 5S .

So that’s it – hope you find your own way to create secure and memorable password folks !
Last but not least, even if you come up with the most difficult passwords to crack, you’re still prone “social engineering” !

Lucene vs. Solr vs. ElasticSearch vs. Sphinx ? How to do real-time full text search properly ?

Currently I am researching on the solution for a real-time search engine which scourge user submitted contents. The user pool consists of 250+ private institutions which translate into 10K-15K users. I/O will be relatively low, but the data size could be ranging wildly since the users are uploading Microsoft Words, Excel and PDF file.

The index is based on the user uploaded files (mostly are Word/Excel/PDF/PowerPoint, and ASCII files). The I/O is expected at only 10 IOPS -20 IOPS but it can vary depends on the date. Maximum I/O could be 100 IOPS. Current database size is reaching 10GB, it's 4 months old.

For real time search server, I'm considering Solr / Lucene and probably ElasticSearch. But the challenge is how to index these files FAST, so that search server can query the index in real time.

I have found some similar questions on how to index .doc/.xls/.pdf, but they did not mention how to ensure indexing performance:

• Search for keywords in Word documents and index them
• Index Word/PDF Documents From File System To SQL Server
• How to extract text from MS office documents in C#
• Using full-text search with PDF files in SQL Server 2005

How to build the index FAST ?

Any suggestion on the architecture ? Should I focus on building fast infrastructure (i.e. RAID, SSD, more CPU, Network bandwidth ?) or focus on the index tools & algorithm?

P/S: this question is asked on Stackoverflow as well

Finding the best of the breed – or which one to rule them all ?

Disclaimer: this article not intended to ignite flames or cause arguments on the Internet. It is about evaluating the candidate languages or frameworks to use in the N-tier programming model (from presentation tier, logic tier to data tier), and there is no one-size-fits-all. So, please take it with a grain of salt and read on.

Presentation tier (front-end)
For Javascript: My love goes to jQuery: there are many alternatives such as Prototype, MooTools, Scriptaculous but none can beat jQuery in terms of documentation or community support. Moreover, many CDNs support jQuery.

For CSS, jQueryUI or Bootstrap are my favourites. For big project, I’d go with jQueryUI – reason being is jQueryUI offer many features more than Bootstrap (Accordions, Draggable ..etc).  For small projects, I’d go with Bootstrap (very lightweight and quite comprehensive). Of course, I’d write CSS using SASS ! You may ask why not LESS: well, here and here are some supporting evidences.

Logic tier (middleware & API)
This one is tough – every language and framework can be used to produce effective middleware & API code depending on your business requirement and usage scenarios. Therefore, I would not boast about any name. Instead, there are some good principles and practices for writing efficient API code. Of course there are some great books on this matter that I'd wholeheartedly vouch for : Clean Code, The Pragmatic Programmer.

Data tier (storage, database & file server)
In today’s distributed computing world, Hadoop is one of the champions (in IMHO) – I found these tips are pretty useful before you’re going to adopt Hadoop. I also heard about Hive and its usage in Linkedin datacenter but never used Hive in any of my projects. I’d love to have a chance to experience it in production. Nevertheless, here is a fair comparison: Hadoop vs Hive (courtesy of Zheng Shao - Facebook Engineer Manager)

For storage – NoSQL is a rising star vs traditional SQL. Big-data companies (Facebook, Twitter, LinkedIn, Google, Yahoo) adopted NoSQL long ago – so it’s time you learn about it, join me in learning NoSQL and share your experience with me :)

Right now, I am starting with MongoDB with a MEAN stack. Man I’m loving the speed of this stack compared to ASP.NET MVC or even LAMP/XAMPP.

Other notes
Javascript alternative (altJS): TypeScript, CoffeeScript, Dart.
Once upon a time, writing several hundred lines of Javascript would be a nightmare – no longer the case when there is Typescript. It lets you write classes and declare namespace (for code cohesion), implement inheritance (IMO, one of the most useful O-O features).


So that is a few lines from my thoughts on a bright sunshine Monday –  do you agree, do you not ? Let me know and we can always discuss ;) of course not in this manner ^_^

Defensive programming – simple method to prevent Object reference not set exception

 

Just to share a simple way to prevent the ugly “object reference not set” exception: 

public class DefensiveCode
    {
        //Non-defensive code (unsafe):
        public IList<string> UnsafeProps { get; set; }
        public void DoSomething(string input)
        {
            //props can be null --> exception
            if (UnsafeProps.Contains(input))
            {
                //....do work
            }
        }

        //Defensive code #1:
        public IList<string> SafeProps { get; set; }
        public void DoSomethingSafely(string input)
        {
            //a little better, but have to check every time
            //and it may depend on how compiler evaluate: L-to-R is OK, but R-to-L will not be OK
            if (SafeProps != null && SafeProps.Contains(input))
            {
                //....do work
            }
        }


        //Defensive code #2: (preferred)
        private IList<string> _SafestProps;
        public IList<string> SafestProps
        {
            get
            {
                //initialize before accessing it. 
                if (_SafestProps == null) _SafestProps = new List<string>();
                return _SafestProps;
            }
            set { _SafestProps = value; }
        }

        public void DoSomethingSafestWay(string input)
        {
            //it’s always safe
            if (SafestProps.Contains(input))
            {
                //....do work
            }
        }
    }

A lesson learnt: do not return NULL , return empty list or empty string instead.