Add this line in the end of '/etc/profile' file:
Problem solved! Make sure $HOME is pointing to your home directory and you have .bashrc file generated.
[Update] Cygwin always load off of '.bash_profile' instead of '.bashrc' file. Thus, another solution is to do 'source ~/.bashrc' within '.bash_profile'. Or add the following line in '.bash_profile':
if [ -f ~/.bashrc ]