The Lord is the one who is to do the judging, and the casting off of the branches/trees that are unfruitful, but we can KNOW them by their fruits, in other words... we can
know by the fruit being produced if one is producing good or evil fruit .... but the casting off and cutting down, and casting into the fire is the Lord's business, not ours.
Matt. 7:
1 Judge not, that ye be not judged.
2 For with what judgment ye judge, ye shall be judged: and with what measure ye mete, it shall be measured to you again.
16 Ye shall know them by their fruits. Do men gather grapes of thorns, or figs of thistles?
17 Even so every good tree bringeth forth good fruit; but a corrupt tree bringeth forth evil fruit.
18 A good tree cannot bring forth evil fruit, neither can a corrupt tree bring forth good fruit.
19 Every tree that bringeth not forth good fruit is hewn down, and cast into the fire.
20 Wherefore by their fruits ye shall know them.