Don't threaten anything. If you are going to sue someone, never give them notice.
He needs to perfect service if he wishes to have a judgment that will hold up in court (this means delivering a notice to your place of residence, or obtaining proof that the notice to appear was delivered to you). He is not doing this.
Even if there is a hearing (which I doubt will happen) where you don't show up, he will still need to present his case. If he has no case, the magistrate (not a judge in small claims) will throw it out.
On the funny side, if you go to court and the judge sides with you, then you can ask for a restraining order since he has been harassing you about paying for something which has nothing to do with you. Him being present at the hearing is notice of the restraining order, so it takes effect immediately. I was going to have one issued to the collection agency I sued.