In fact, rereading Frege's piece, I was greatly impressed not only by its seriousness and intellectual fecundity but also by the careful and respectful way Frege – who was basically a philosophically oriented mathematician and logician – dealt with issues of natural language.
He did not draw easy conclusions but was often content simply to sketch out the semantic complexities involved in certain kinds of syntactic contexts, for example in relation to reported speech or subordinate clauses involving the subjunctive mood.
What has always been clear to me (and which this recent discussion only confirms and underscores) is that aspects of Frege's work have been appropriated and incorporated into a (philosophical) project and a narrative which is quite alien to what Frege – who identified as a man of science – saw himself to be doing.
The comment thread is interesting for a number of reasons but particularly perhaps for the contribution towards the end by someone who has recently translated some early work by Frege and who suggests – as I am suggesting here – that Frege would have been quite out of sympathy with most late-20th century philosophers of language, even those who claim him as a precursor.
I don't want to suggest that the science/non-science distinction is clearcut. It's not. For one thing, there are obviously huge differences between empirical work in different areas (between empirical work in physics and the study of natural language, for example); and also an important distinction between empirical work in general and the building of mathematical or logical or conceptual structures which may be intended (as Frege's were) for basically scientific applications.
But clearly certain kinds of discourse, such as aesthetic, moral and ideological discourse, while they may draw on scientific knowledge to a greater or lesser extent, are not scientific – and (contingent of course on one's conception of philosophy) not necessarily 'philosophical' either.
My first comment on the thread (which I quote here in a modified form) touched on this point.
[Quoting another commenter] "... So it seems Frege is really excluding discussions that clarify or debate sense [as distinct from reference] ... from logical analysis. (This is certainly in keeping with the agenda of Logical Positivism that followed after, which held that topics in aesthetics, religion, politics, etc., were philosophically moot.)"
Yes, this is a very important point. The references to Homer's Odyssey etc. in the Frege's paper seem to confirm what you are saying. Frege obviously respected poetry and fiction etc. but took discourse relating to these things, other hypotheticals and also political rhetoric as being in a very different category or set of categories from the kind of discourse and inquiry (scientific, philosophical) in which he saw himself to be engaged...
Frege's attitude to signs which have a sense but not a reference, which of course include not only the names of fictional characters like Odysseus but also many other kinds of expression and which he tentatively refers to as 'representations', is ambivalent. He accepts that they are an essential element of natural language-based communication, but sees them as operating beyond the scope of science and the sort of logical enquiry he is interested in, except in the sense that the latter may provide certain salutary lessons.
In the vast majority of cases the use of such 'representations' is perfectly fine, but sometimes problems can arise which the logician may be able to diagnose or at least see in a perspicuous way. Frege warns, for example, about political contexts in which expressions like "the will of the people" – which have a sense but not a reference (or at least not any clear or agreed one) – are used rhetorically for purposes of social or political manipulation.