Venturing into the internet

Thread Starter

Ian0

Joined Aug 7, 2020
9,667
I've been programming microcontrollers for many years, but I have never attempted to connect them to the internet; and I'd like to move out of my comfort zone and attempt a new challenge.
Modules are available for very low prices using W5500, ENC28J60, ESP8266 etc. and I'd like to find out how to communicate between them on a local network and the wider www, and understand the difference between UDP, TCP, Telnet etc.
There are plenty of join-the-dots projects out there, but I need to understand what I am doing.
Where do I find the information I am looking for? Are there any good books? Websites? etc. What does anyone recommend?
 

ericgibbs

Joined Jan 29, 2010
18,766
hi Ian,
On your wish list add the ESP32 Wroom module, it is well covered on Youtube videos for your proposed projects.
E

Added a video, 'Andreas' is an excellent source of information

 
Last edited:

BobTPH

Joined Jun 5, 2013
8,804
I was in the same position about a year ago and took the leap. I use PIC24/33 processors and I have my own compiler, so I wanted to continue with them.

My first try was a microchip WIFI board, which was supprted by libraries for PIC. I found it unacceptable because took over 128K of code space on the host processor.

So I latched on to the ESP-01 board, which has an ESP8266 and all firmware to access TCP/IP pre-loaded. It communicates with the host over serial line at 115K which is fine for my app, but would not be enough for something like streaming video. It is tiny, about 1x2 cm, and I piggybacked it on my PIC24 board.

As far as learning the interface, I found this site:

W3Schools

invaluable for its tutorials on TCP, HTTP, and HTML. And they are free.
 

ApacheKid

Joined Jan 12, 2015
1,533
I think it comes do supporting sockets, if there are sockets then you can implement HTTP and if you can implement HTTP you can implement a web server.
 

Thread Starter

Ian0

Joined Aug 7, 2020
9,667
I think it comes do supporting sockets, if there are sockets then you can implement HTTP and if you can implement HTTP you can implement a web server.
"When I use a word," Humpty Dumpty said, in rather a scornful tone, "it means just what I choose it to mean—neither more nor less." and I suspect that in this context a socket isn't a receptacle into which a plug fits, but give me a week and I will have probably caught up.
 

ApacheKid

Joined Jan 12, 2015
1,533
"When I use a word," Humpty Dumpty said, in rather a scornful tone, "it means just what I choose it to mean—neither more nor less." and I suspect that in this context a socket isn't a receptacle into which a plug fits, but give me a week and I will have probably caught up.
Yes, its actually a rather primitive abstraction and poorly named, very little resemblance to any physical socket concept! A socket is a local thing, an app creates a socket by either "listening" for incoming connections or by making an outgoing connection request.

One you have your socket you can send and receive contiguous block of bytes, raw bytes basically, with some length.

The TCP/IP stuff under the hood ensures in-order reception of "packets".

A web server is then just an app listening for incoming socket connections, then when it gets a connection it reads what the sender sends which should be (ultimately) a message that conforms to the HTTP rules, with a type, headers etc.

A browser sends a "GET" HTTP message and the message includes URLs and so on, the web server's job is to decompose that message and find the thing/data referred to by the URL and then simply send that back over the socket, the remote browser end simply waits for that response.

Once the response is sent and received, each end closes the socket and the web server goes back to waiting for another incoming connection.
 

Irving

Joined Jan 30, 2016
3,843
Network Programming for the ESP32 is relatively straightforward, and while having understanding of the underlying plumbing can be helpful its not essential. You don't need sockets to do many things, there's a lot of abstraction built into libraries, whether you use the Expressif ones or the ported Arduino ones. The ESP32 examples in the Arduino IDE 'Examples' folder are worth looking at.

I offer, by way of example, my own simple UDP broadcast and UDP receive that serve as the basis of a number of fire & forget data transmission within my home environment - monitoring the status of something doesn't always require the complexity of a handshake - just knowing there should be a transmission at least once a second or so is sufficient to flag if somethings stopped working.
 

Attachments

Thread Starter

Ian0

Joined Aug 7, 2020
9,667
I was in the same position about a year ago and took the leap. I use PIC24/33 processors and I have my own compiler, so I wanted to continue with them.

My first try was a microchip WIFI board, which was supprted by libraries for PIC. I found it unacceptable because took over 128K of code space on the host processor.

So I latched on to the ESP-01 board, which has an ESP8266 and all firmware to access TCP/IP pre-loaded. It communicates with the host over serial line at 115K which is fine for my app, but would not be enough for something like streaming video. It is tiny, about 1x2 cm, and I piggybacked it on my PIC24 board.

As far as learning the interface, I found this site:

W3Schools

invaluable for its tutorials on TCP, HTTP, and HTML. And they are free.
Do you drive it with the AT commands?
 

Ya’akov

Joined Jan 27, 2019
9,069
I had a look and got completely distracted by a silly video about bicycles.
One of the things I like about the channel is that the speaker is often someone who “was there”. They have people who participated in building the tools and standards we use today, and they have great insight.
 

ApacheKid

Joined Jan 12, 2015
1,533
I had a look and got completely distracted by a silly video about bicycles.
I explored the ESP8266 a few years ago when I started getting interested in MCU stuff. I wrote an API in C# to help me experiment and understand the device. The idea was (after getting some solid code working) to recode that C# in C and have some confidence that the code was at least managing the device correctly.

It is much easier to explore the ESP8266 by just plugging it into a PC and running the test app there, C# is a better language too so one can reach a good level of quality.

In the end (I never did quite finish) I had this produced

  • C# app and decent class library abstracting the ESP8266
  • Ability to plug an ESP8266 into a PC and manage it
  • Ability to use the ESP8266 to create, open and communicate over an ESP socket.
  • Ability to transfer data from that test code to a pure .Net app running as the other end of a socket.
  • Initial migration of the C# code to C and ability to do socket IO from an STM discovery board.

The C# code is intended only for running on a PC and delivers a high quality test/debug capability allowing once to fully unravel the serial IO AT commands.

Here's the repo, in principle this will just work if you plug an ESP8266 into a PC and have the little driver that's needed, the root class for the device is here.

As I said the purpose of this code is to provide a high quality test environment for helping to understand and test an ESP8266 without the added frustration of testing it on a real MCU.
 

Thread Starter

Ian0

Joined Aug 7, 2020
9,667
I like the idea of AT commands - it suits the rest of the system. I'm using a Renesas R7FA4M1 and I have plenty of SCI ports spare. I also would like to avoid another device that has to be programmed with yet another programming tool. The ESP32 is impressive, at the moment it looks to me like a solution to a problem that I have not yet come across, it wouldn't be the easiest to incorporate in the present design as my data is on CAN.
I will probably end up designing this three times, once for GSM, once for Wi-Fi and once for wired ethernet, so if I can find three modules that connect using AT commands, I don't also have to design the hardware three times.
 

Thread Starter

Ian0

Joined Aug 7, 2020
9,667
There seems to be quite a range of different ESP32s. I'm not sure if it's the same chip mounted in different modules or different varieties of the chip. Is the AT-command addressing built in to all parts or is it something that some module-producer has programmed it to do?
 
Top