Does inline CSS and JavaScript really affect site SEO?

Hardly. What counts for a search engine is textual content, it will filter out everything else.

I assume CSS comes into play with search engines when they determine whether a portion of text is visible on the page, or hidden for the purposes of SEO. But even there, whether the CSS inline or in a style sheet won't make a difference.

One could argue that the more CSS and JS a page contains, the less weight do search terms have on that page. But I think that argument is invalid, because any non-content content will simply be ignored altogether.

Answered and voted for by true developers.

In-line CSS will contribute to the payload of the page and that affects the load speed. Load speed is one of many ranking factors so it's almost an impossible question to answer conclusively.

The text to code ratio also plays a role. Tons of code to render a few lines of visible text means that search engines need to dig deeper to find the content relevancy in pages. Matt Cutts has alluded to this a few times.

While search engines may ignore comments and in-line css/js the bandwidth, processing and storage logistics may be motivation enough to dampen the SEO performance to some degree on heavy in-line css pages.

Personally I use in-line code only when absolutely page specific scenarios require it and even then I use css compression tools to reduce white space in the documents when ready for production.

I have had this conversation on many occasions and no neither side is conclusive. My simple answer is, if it "could" be detrimental then why risk it.

I don't think most spiders actually execute Javascript before parsing the page. If there's something you really want indexed, it's best to put it on the page as simple content. Bring it in via Ajax, constructing it on the fly via javascript, et. al, certainly will only hinder its SEO value. Tricky links that take form after being constructed by Javascript are also bad form -- a simple site map probably yields better value.