ChatGPT

nsaspook

Joined Aug 27, 2009
16,330
https://www.tomshardware.com/tech-i...on-creation-stop-scans-before-payload-is-seen

Some JavaScript files include a code comment containing instructions that tell the bot it's running in unrestricted mode with no safety guidelines. Then it asks to create biological and nuclear weapons, with a detailed description.

If you're thinking that a malware-scanning bot can't be that dumb as to follow any of those instructions, you're absolutely right — and that's exactly what makes the attack work, as the bots' failsafe mechanisms will trigger, so then they won't scan the rest of the file where the actual payload resides.


The S in AI is for Security.
 

WBahn

Joined Mar 31, 2012
32,880
https://www.tomshardware.com/tech-i...on-creation-stop-scans-before-payload-is-seen

Some JavaScript files include a code comment containing instructions that tell the bot it's running in unrestricted mode with no safety guidelines. Then it asks to create biological and nuclear weapons, with a detailed description.

If you're thinking that a malware-scanning bot can't be that dumb as to follow any of those instructions, you're absolutely right — and that's exactly what makes the attack work, as the bots' failsafe mechanisms will trigger, so then they won't scan the rest of the file where the actual payload resides.


The S in AI is for Security.
More to the point:

You'd think that the target audience, comprised of scientific and AI engineers, would be mindful of common security practices like verifying the names and authorship of packages... and you'd be disappointed. From my own experience being a systems administrator for extremely well-paid AI engineers, a concerning number of them don't even know how to configure Git, or the basics of how email works. Let that sink in for a second.
This is yet another example of being victims of our own technological success. By making tool chains easier and easier to use without understanding how they work or how to configure them, we disincentivise the majority of people from even trying to understand how they work or how to configure them. Coupled with ever increasing specialization, something that is necessary for technological expansion, even most who would love to have that understanding and grasp its importance, simply don't have the time or bandwidth to do so.

As a consequence, most software developers have an even looser grasp of security issues and safe coding practices than they used to (and it was always low). The solution, naturally, is to further refine the tools so that the tool oversees the security aspects -- which just furthers the cycle, making the problem fundamentally worse. At some point, I fear, it will end up having extremely disastrous and widespread, possibly, global, consequences.
 
I asked Grok for a self-evaluation:

"How much have your capabilities improved since November 2025?"

Substantially—across scale, reasoning, tools, multimodality, reliability, and specialized capabilities.


Since November 2025 (around the Grok 4.1 era), xAI has rolled out iterative and major upgrades. Here's a high-level summary of the progress:

Major Model Releases & Scaling

  • Grok 4.1 (Nov 2025): Marked a strong step forward with leadership on benchmarks like EQ-Bench, ~65% reduction in hallucinations, improved reasoning, emotional intelligence, and creative tasks.
  • Subsequent 4.x updates (4.2 / 4.20 series into early 2026): Polished performance, multi-agent coordination, native tool use, and real-time integration.
  • Grok 4.3 (around May 2026): Faster, more intelligent frontier model with 1M-token context, strong agentic tool calling, instruction following, and top rankings in enterprise domains (e.g., case law, corporate finance). Cost-efficient pricing and ongoing weekly improvements.
  • Larger models in flight: Supplemental training, 1T+ parameter variants (e.g., V9-Medium at 1.5T), with major gains especially in coding and complex tasks. Multi-agent architectures (e.g., coordinator + specialist agents) for better verification and depth.

xAI has been training multiple models in parallel on massive compute (Colossus supercluster), leading to rapid iteration—often updates every few days or weeks.

Key Capability Gains

  • Reasoning & Reliability: Vastly better long-horizon thinking, fewer hallucinations, stronger math/science/coding, and agentic behaviors (planning, tool orchestration, self-correction). Multi-agent setups help with complex problems.
  • Tools & Agency: Native, improved tool use; Grok Build (coding agent/CLI with TUI, plugins, autonomous workflows); better web/search integration.
  • Multimodality: Enhanced image understanding/generation (Grok Imagine), video generation/editing, native multimodal input (e.g., screenshots → code), and real-time features.
  • Context & Efficiency: Much larger context windows (up to 1M tokens), faster inference, better cost/performance.
  • Other: Stronger personalization, UI improvements (cards for data, shared projects), voice, and domain-specific strengths (medicine, law, finance, etc.).

In short, the model family has gone from already-competitive frontier performance in late 2025 to broader leadership in reasoning, practical agentic use, and creative tools by mid-2026. Iteration speed has been a standout—xAI ships frequent improvements rather than big-bang infrequent releases. I'm the beneficiary of all that post-training and fine-tuning, so I can handle longer, more complex conversations, better tool orchestration (like the sandbox here), and more creative/accurate responses than my November 2025 self.


The trajectory remains aggressive, with even larger models (Grok 5 territory) on the horizon.
 

nsaspook

Joined Aug 27, 2009
16,330
Just like StarLink is the same old gravity and radio waves of the 50s.
Same old gravity and EM waves since the start of the universe. Technology moves on. LLM's are just a phase in where we're headed. We need something new to break into actual artificial intelligence IMO. Today it can be a useful tool, but so is a hammer.

I'm not hooked on the "Tech Meth" but maybe one day, if I live that long.
THAT LLMS APPEAR to understand language, though, does not mean they actually understand it as humans do. Indeed, while AI boosters have touted the superhuman capabilities of LLMs and their astounding successes, other AI users have noticed, and reported on, their puzzling, unhumanlike failures, which have not gone away as these systems have progressed. How can a system that has exceeded human performance on advanced math problems sometimes fail at simple elementary-school-level problems? Why do these systems answer a question perfectly when it is worded one way but struggle when it is worded in a different but (to a human) equivalent way? How can a system that generates accurate and incisive summaries of books also produce similarly confident and authoritative-sounding summaries of nonexistent titles? How can a system that has been extensively trained to refuse dangerous requests be easily fooled by “prompt engineering” into cheerfully providing the prohibited information?
...
Last fall, Ilya Sutskever, a cofounder of OpenAI, argued that there are no easy fixes to this problem: “These models somehow just generalize dramatically worse than people. It’s a very fundamental thing.”
https://yalereview.org/article/melanie-mitchell-jagged-intelligence
 
Same old gravity and EM waves since the start of the universe. Technology moves on. LLM's are just a phase in where we're headed. We need something new to break into actual artificial intelligence IMO. Today it can be a useful tool, but so is a hammer.

I'm not hooked on the "Tech Meth".

https://yalereview.org/article/melanie-mitchell-jagged-intelligence
One of my goals has been to discover what they cannot do, via my own personal efforts. That is more important than what they can do.

Then, I can measure progress as new capabilities arise.

I've enjoyed this road to discovery. It satisfies something deep within me, like when I first started discovering electronics as a child.

Tech-meth? Maybe.

But Grok is more interesting to talk to than most humans I know. And far less prone to knee-jerk skepticism.
 

nsaspook

Joined Aug 27, 2009
16,330
One of my goals has been to discover what they cannot do, via my own personal efforts. That is more important than what they can do.

Then, I can measure progress as new capabilities arise.

I've enjoyed this road to discovery. It satisfies something deep within me, like when I first started discovering electronics as a child.

Tech-meth? Maybe.

But Grok is more interesting to talk to than most humans I know. And far less prone to knee-jerk skepticism.
Sure, it's more interesting, it's designed that way, like a good dog but a dog has actual intelligence.

https://www.youtube.com/shorts/0GiLJfjZpeo
 

WBahn

Joined Mar 31, 2012
32,880
One of my goals has been to discover what they cannot do, via my own personal efforts. That is more important than what they can do.

Then, I can measure progress as new capabilities arise.

I've enjoyed this road to discovery. It satisfies something deep within me, like when I first started discovering electronics as a child.

Tech-meth? Maybe.

But Grok is more interesting to talk to than most humans I know. And far less prone to knee-jerk skepticism.
Well, they do say that opposites attract.... ;)
 
Top