Ubuntu 14.04 keeps asking for passphrase when ssh

I have ubuntu 14.04 workstation installed in my LAB. Now I have issue with rsa key passphrase.

Following are what I have in my LAB

• SVN server(12.04): I have svn server installed, anyone who want to access svn must have their key in authorized_keys file of svn server.
• Home Server(12.04): It's mounted as /home on clients
• Ubuntu Desktop 12.04: There are more than 5 clients connected to my ubuntu lab (using for many years)
• Ubuntu Desktop 14.04: I have just installed 14.04 version for test

This my issue,

• From Ubuntu Desktop 12.04 it doesn't ask for passphrase but Ubuntu 14.04 does. It keeps asking for passphrase everytime I use SVN command.
• I tried recreated rsa key but it doesn't work.
• I test created key without passphrase, it doesn't ask me to enter passphrase. And it isn't what I want because unsecured.

Any ideas?