To allow only some specific users/groups access the SSH server: $ sudo nano /etc/ssh/sshd_config Add the following line to allow access only for user 'trinh': AllowUsers trinh Or this line to allow only group 'mygroup': AllowGroups mygroup Restart ssh service: $ sudo service ssh restart More options (source: http://knowledgelayer.softlayer.com/learning/how-do-i-permit-specific-users-ssh-access) : AllowGroups This keyword can be followed by a list of group name patterns, separated by spaces.If specified, login is allowed only for users whose primary group or supplementary group list matches one of the patterns.`*' and `?' can be used as wildcards in the patterns.Only group names are valid; a numerical group ID is not recognized.By default, login is allowed for all groups. AllowUsers This keyword can be followed by a list of user name patterns, separated by spaces.If specified, login is allowed only for user names that match one of the ...