I actually view the use of AI and the use of calculators as being quite decent parallels at a pretty fundamental level -- AI just brings those problems to the table on steroids.https://cacm.acm.org/opinion/generative-ai-and-cs-education/
Generative AI and CS Education
Increased knowledge sharing is helping CS educators and researchers accelerate change in computing education.
I was in grade school in the pre-calculator era. When calculators/computers became cheap and usable we still had the "number sense" to use them as tools (as working agents of our knowledge) instead of a crutch. I'm watching a daughter take college level programming courses in C++ and now C (using K&R as the course book) for her computer architectures classes while taking calculus 3 and physics. So far Generative AI has only been lightly touched during her learning process so they can develop the 'code sense' of a traditional CS education. Integrating AI assistants into education will IMO be totally different than the use of calculator as unless there is a calculation error, they don't hallucinate answers. The "AI" gives you an answer (without understanding) and tries hard to convince you it's correct (at face value). You can only tell whether it's a good answer or not if you're capable of writing the good answer yourself.
How to teach the 'code sense' to see and detect these hallucinated and sometimes detailed, at times, very complicated code responses from autocomplete chatbots will be an interesting process. These tools deliver code but writing code has very little to do with "computer science".
IMO current LLMs are not early prototypes, they are pretty much at the limits of that's capable with that technology. The reliability and trust problems we see today likely can't and won't be fixed long-term with LLM based programming.
As you stated, AI gives you an answer and you can't tell if it's a good answer or a bad answer if you are not capable of writing the good answer yourself -- or at least have some basis upon which to evaluate the quality of the offered answer. But the exact same is true with a calculator. It gives you an answer and if you do not have any basis upon which to evaluate the quality of the offered answer, you are highly likely to simply accept whatever it gives at face value. I see that happen all the time.
I recently gave an exam in a computer networks class in which they were given a problem in which realistic values for link length, transmission rate, and packet size were given and they were asked to determine the effective data transfer rate if a Stop and Wait protocol were used (assuming zero processing and queuing delays). I got answers that ranged from many (as in dozens) of orders of magnitudes faster than the link transmission rate, to answers so small that it equated to less than one bit transferred over the current age of the known universe. This is despite having harped on tracking units (which hardly anyone did) and asking if the answer makes sense (which even fewer did) all semester. The sad fact is that the overwhelming majority of students have gotten to this point with zero number sense and are so reliant on calculators to do their thinking for them that they are not capable of evaluating whether the answer it spews forth is reasonable of not. Even if they were willing to deal with this deficiency (and that number is extremely small, though not quite zero), they have dug themselves such a deep whole (with the educational system being ever ready to give them bigger and bigger shovels to dig with), that it would be a major step back for them to remediate it.
The exact same thing has happened with all kinds of technological advances that are intended to assist human endeavors but, almost invariably, have the effect of substituting for basic human thinking. There is absolutely NO reason to believe that AI is going to be ANY different -- in fact, there is every reason to expect that it will race down that inevitable path faster than anything we have seen before. We can also expect this process to be sped up by the educational system that, like the calculator and other tools before it, will merely latch onto yet another way to "bring technology into the classroom" and turn it into the biggest shovel yet for students to dig themselves holes in which they have to think even less.




