I can't think of a reason as to why that'd be the case, but if you give me a minimal way to reproduce this issue (starting from Emacs -q describe all the steps I need to take including what packages I should install and how I should configure them), I can take a look.

Ok here's a sequence I can run to recreate it; hopefully this is helpful.
This is emacs 26.3 on a Mac via homebrew (the 'cask' version)

• install 'ag' from elpa or whatnot (it lists dash-2.8.0, s-1.9.0, and cl-lib-0.5 as requirements). The 'ag' bin probably needs to be installed via homebrew/apt/etc. I am seeing ag version 2.2.0
• launch emacs via emacs -q
• load-library ~/.emacs.d/elpa/dash-20200426.2244/dash.el
• load-library ~/.emacs.d/elpa/s-20180406.808/s.el
• load-library ~/.emacs.d/elpa/ag-20190726.9/ag.el
• run ag, enter a search term and a directory. This should give a results buffer with entries that can be jumped to via next-error/etc.
• load-library ~/.emacs.d/elpa/xterm-color-20200521.114/xterm-color.el
• eval the code from the docs to filter compilation output:

  (setq compilation-environment '("TERM=xterm-256color"))
  
  (defun my/advice-compilation-filter (f proc string)
    (funcall f proc (xterm-color-filter string)))
  
  (advice-add 'compilation-filter :around #'my/advice-compilation-filter)

• run the same ag command again
• for me, attempting to jump to results now gives a Find this error in (default *unknown*): prompt.

The problem is that ag.el relies on ANSI color codes being part of the input since that's what it uses to match on. From ag-filter (ag.el:632):

(while (re-search-forward "\033\\[30;43m\\(.*?\\)\033\\[0m\033\\[K" end 1)
            (replace-match (propertize (match-string 1)
                                       'face nil 'font-lock-face 'ag-match-face)
                           t t)))
        ;; Add marker at start of line for files. This is used by the match
        ;; in `compilation-error-regexp-alist' to extract the file name.
        (when ag-group-matches
          (goto-char beg)
          (while (re-search-forward "\033\\[1;32m\\(.*\\)\033\\[0m\033\\[K" end 1)
            (replace-match
             (concat "File: " (propertize (match-string 1) 'face nil 'font-lock-face
                                          'compilation-info))
             t t)))
        ;; Delete all remaining escape sequences
        (goto-char beg)
        (while (re-search-forward "\033\\[[0-9;]*[mK]" end 1)
          (replace-match "" t t))))))

This is not going to work when the input is filtered through xterm-color.el, since that translates all ANSI escape sequences into properties. It's also a bad idea to hardcode
this sort of assumption in ag.el, so I would definitely advise its developer to find a better way to
do matching.

I don't use it myself and I'm not going to dig any deeper but there may be an easy way to configure it so that it doesn't rely on ANSI escapes. You should ask its developer/maintainer. I'm closing this issue.

Ok, thanks for taking a look at that. I'll forward the info along to the ag.el maintainer.