SSH DenyUsers

This option will provide possibility for management of access for Users that are connecting to your server from particular IP.

In this way it's possible to limit access on application level.
So far I know it's not possible to do this kind of user access filter on firewall level.

Location: “/etc/ssh/sshd_config”
I personally prefer to put this rules at end of the file to make it readable.

This keyword can be followed by a list of user name patterns, separated by spaces. Login is disallowed for user names that match one of the patterns. Only user names are valid; a numerical user ID is not recognized. By default, login is allowed for all users. If the pattern takes the form USER@HOST then USER and HOST are separately checked, restricting logins to particular users from particular hosts. The allow/deny directives are processed in the following order: DenyUsers, AllowUsers, DenyGroups, and finally AllowGroups. All of the specified user and group tests must succeed, before user is allowed to log in.

See PATTERNS in ssh_config(5) for more information on patterns.

You can put each user in to separate line

DenyUsers user1@*                 # Deny access for "user1" from ANY IP
DenyUsers user2@10.0.0.1          # Deny acesss for "user2" from 10.0.0.1 IP
DenyUsers *@127.0.0.1             # Deny access for ANYONE  from 127.0.0.1 IP

Or you can put all access rules in to one line

DenyUsers user1@* user2@10.0.0.1 *@127.0.0.1

/etc/init.d/sshd restart         # For example at CentOS 6
systemctl restart sshd           # At OS with systemd for example CentOS 7

[root@SSH_SERVER ~]# adduser user1      # Create user1 account
[root@SSH_SERVER ~]# adduser user2      # Create user2 account
[root@SSH_SERVER ~]# passwd user1       # Set password for user1
...
[root@SSH_SERVER ~]# passwd user2       # Set password for user2
...

[root@SSH_SERVER ~]# vi /etc/ssh/sshd_config                                     # Put your rules at the end of this file
....
DenyUsers user1@*                 # Deny access for "user1" from ANY IP
DenyUsers user2@10.0.0.1          # Deny acesss for "user2" from 10.0.0.1 IP
DenyUsers *@127.0.0.1             # Deny access for ANYONE  from 127.0.0.1 IP

[root@SSH_SERVER ~]# systemctl restart sshd                                       # Restart SSHD so that the change will take efect

[user@SSH_CLIENT ~]$ ssh user1@SSH_SERVER            # Use rulle "DenyUsers user1@*"
user1@SSH_SERVER's password:
Permission denied, please try again.
user1@SSH_SERVER's password:
Permission denied, please try again.
user1@SSH_SERVER's password:
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[root@SSH_SERVER ~]$ ssh root@127.0.0.1            # Use rulle "DenyUsers *@127.0.0.1"
root@127.0.0.1's password:
Permission denied, please try again.
root@127.0.0.1's password:
Permission denied, please try again.
root@127.0.0.1's password:
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[user@SSH_CLIENT ~]$ ssh user2@SSH_SERVER            # Use rulle "DenyUsers user2@10.0.0.1"
user2@SSH_SERVER's password:
[user2@SSH_SERVER ~]$