Sharing filters - remove a user

In quick filters, when you view the list of users with whom the filter is shared, it is not possible to remove a user.  It would be nice to click on a name or names to delete them. The only way to remove a user now is to create the filter or copy it and share the new filter with the proper people.