Uploading program over the air ESP32 using the HTTP webserver

geekoftheweek

Joined Oct 6, 2013
1,201
Glad to hear... may have to try that myself.

I just managed to get it to work for me and was checking back to your screen shot of the memory settings to see if I could offer any more help. I had to select what you already did memory wise to get mine to work. Strange it worked for me and not you. Maybe the NodeMCU has something different set up that mine does not and erasing the flash reset things.

At any rate happy to hear you finally sorted it out!!
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
Hey im back here again! :D I have switched to raspberry PI 4 and reconfigured it as I did with my raspberry PI 3. All of a sudden now, i cannot get the OTA to work anymore. I am able to upload files to my http server on raspberry PI, however when Im trying to do an OTA:
1598939658021.png

I am initiating OTA;
1598939679836.png
I have confirmed that the chipid.bin is in uploads folder. Why does it the ESP32 does not see that file?
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
Problem solved! The binary files must have to be stored in /var/www/html/uploads and I previously had them stored in /var/www/uploads
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
Hey its me again. I have recently received a NAS drive and was wondering if I can do an OTA directly from the .bin file that is stored on my NAS drive instead of my Raspberry PI webserver, I am currently calling httpupdate function as following:

t_httpUpdate_return ret = httpUpdate.update(client, mqtt_server,80,"/uploads/PTL.bin");//esp32
mqtt_server - refers to the IP of the raspberry PI

I have mounted my NAS storage as a drive and can access it on my raspberry pi location:

/nfs/temp-share


What is the easiest way to change the httpUpdate to do an OTA from any other location than /var/www/html/uploads
 

geekoftheweek

Joined Oct 6, 2013
1,201
@zazas321 the easiest way I can think of is creating a symbolic link.
Code:
in -s /nfs/temp-share /var/wwe/html/uploads
should create a temo-share directory you can access from web server. There are a couple options you could configure in apache, but i'm not home to look them up.
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
Thanks for the fast response! I have created a shortcut to my NAS drive inside /var/www/html/uploads
with the command:
ln-s /nfs/temp-share/PTL_binary /var/www/html/uploads

and I can see the folder there.

However, when i try to do an OTA from the shortcut folder, it fails:

Code:
t_httpUpdate_return ret = httpUpdate.update(client, mqtt_server,80,"/uploads/PTL_binary/PTL.bin");//esp32
1601024681055.png


However, when using my previous OTA method: ( just removing /PTL_binary from the directory location)
Code:
t_httpUpdate_return ret = httpUpdate.update(client, mqtt_server,80,"/uploads/PTL.bin");//esp32
works fine
.

I have confirmed the privilleges on that folder and it seems fine:

2020-09-25-120505_1920x1080_scrot.png




I have also noticed this thread:

https://askubuntu.com/questions/622572/how-to-use-a-symlink-in-apache-web-server
Code:
<Directory /nfs/temp-share/PTL_binary/>
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>
Not sure which syntax for directory is correct though:
<Directory /nfs/temp-share/PTL_binary/>
OR (without an / at the end)
<Directory /nfs/temp-share/PTL_binary>




And added the directroy in my apache2.conf


Unfortunately, that did not make any difference
 
Last edited:

geekoftheweek

Joined Oct 6, 2013
1,201
I was kind of afraid of that happening. I had the same issues sort of with NFS shares at one time. In certain situations symbolic links won't work because it could be used as a way to gain access to higher level directories or directories outside where it is supposed to be.

The thread you mentioned was what I was thinking of at the time with Apache. I think you do need the / at the end. It's been a long time though so I could be wrong. Another option would be to create a virtualhost that has your NAS share as the root directory... say /nfs/temp-share/www. It's a little involved though.

One last option that worked for me looks the same as a short cut / link, but on the system level it gets treated different. In your /etc/fstab add
Code:
/nfs/temp-share /var/www/html/uploads none defaults,bind 0 0
and that will more or less map /nfs/temp-share to /var/www/html/uploads. Move everything out of the /var/www/html/uploads that you need to keep. It won't get erased... you simply won't be able to access it unless you unmount it first (umount /nfs/temp-share). Look up bind mount for more details.

I just thought I should mention if your NAS is something that isn't going to always be accessible you should add noauto to the defaults,bind options and mount it manually using mount /nfs/temp-share. That will make it not automatically happen at boot which can delay things at times.

I can think of a few other options, but it's going to get a bit more complex. Good luck.
 

geekoftheweek

Joined Oct 6, 2013
1,201
I just remembered seeing a previous thread about mounting CIFS at boot and made the connection. The /etc/fstab suggestion above should work without the noatuo. Your fstab on that thread made things make sense.
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
Wow thanks for sharing your knowledge as always! Il try modifying fstab as you suggested
Code:
/nfs/temp-share /var/www/html/uploads none defaults,bind 0 0
And see what it does on Monday

Getting an OTA from NAS would be much more convenient than /var/www/html because I use windows machine to write Arduino code. Everytime I want to do an OTA update, I have to copy over the binary file from my windows machine to my raspberry PI, then add the binary to my /uploads folder.

Doing it from NAS would be much more convenient since I can just drop the binary to the NAS drive and automatically see the file on raspberry since both machines are on the same network and have access to NAS
 

geekoftheweek

Joined Oct 6, 2013
1,201
Wow thanks for sharing your knowledge as always! Il try modifying fstab as you suggested
You are welcome. It beats the hours of digging through information online to find the answers.

I see where this is going now. If for some reason the NAS setup refuses to play nice you could try setting up Samba on the Pi. It's a CIFS server that will allow Windows to share directories. That was actually where I had problems before and found I had to use the bind mount trick to get everything to work.
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
I do not want to bind the whole NAS drive to my var/www/html/uploads just the part of it. NAS mainly will be used to store thousand of pictures and I do not need to access those pictures from my var/www/html/uploads. Therefore, on my NAS drive I have created a separate folder named "PTL_binary" and I store my .bin file there.
1601269085708.png

I have tried to use command :
Code:
/nfs/temp-share/PTL_binary /var/www/html/uploads/PTL_binary none defaults, bind 0 0
Now I have 2 almost identical .bin files. One is in /var/www/html/uploads/PTL.bin and the other one from my NAS Drive /var/www/html/uploads/PTL_binary/PTL2.bin.

2020-09-28-081314_1920x1080_scrot.png

Trying to do an OTA from the NAS binary (PTL_binary/PTL2.bin) results in error:

HTTP error: no HTTP server




Also tried to bind it directly to /var/www/html/uploads (without the extra PTL_binary folder). I can see that it have removed all the files that I previously had and I can only see the binary from the NAS Drive:
2020-09-28-082036_1920x1080_scrot.png

However, that also returns an error:
HTTP error: no HTTP server
 

Attachments

Last edited:

geekoftheweek

Joined Oct 6, 2013
1,201
Something isn't adding up right. Are you using static IPs on everything or DHCP? I'm wondering if an IP address changed and you didn't catch it. I'm thinking the error should be more like "file not found" instead of "HTTP error: no HTTP server". Have you tried accessing the web server from a normal browser? Other than that I'm kind of at the end of my knowledge.
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
Something isn't adding up right. Are you using static IPs on everything or DHCP? I'm wondering if an IP address changed and you didn't catch it. I'm thinking the error should be more like "file not found" instead of "HTTP error: no HTTP server". Have you tried accessing the web server from a normal browser? Other than that I'm kind of at the end of my knowledge.
I am not using static IP's. And im not sure if I could have changed the IP or something since I can access the NAS drive on my raspberry PI either from the IP address or from /nfs/temp-share ( thats where it mounts in fstab). I am already using this NAS drive to store the pictures from my rapsberry PI so I know it works.

I am really confused since both .bin files look exactly the same, however it wont let me use the one that comes from bound NAS drive but let me use the one that I manually place in/var/www/html/uploads
 

Thread Starter

zazas321

Joined Nov 29, 2015
936
Ok ima go through it again make sure I did not make any silly mistake. Il show step by step what I did:

1. In my NAS drive, I have created a folder called "PTL_binary" and i placed a binary file in there "PTL2.bin"

2. In my var/www/html/uploads I have created 2 folders : "PTL_binary_original" and "PTL_binary". Inside "PTL_binary_original" , I have manually copied the .bin file. Inside "PTL_binary" , I use fstab to bind it to my NAS drive
2020-09-30-085335_1920x1080_scrot.png

In my fstab, I have added a line:
Code:
/nfs/temp-share/PTL_binary /var/www/html/uploads/PTL_binary none defaults, bind 0 0
after executing the fstab using sudo mount-a, I can now see "PTL2.bin" inside the /var/www/html/uploads/PTL_binary. It is there at the size of the binary is right, privilleges also seem right.

And as again..
Code:
t_httpUpdate_return ret = httpUpdate.update(client, mqtt_server,80,"/uploads/PTL_binary_original/PTL.bin");//esp32
wroks fine


Code:
t_httpUpdate_return ret = httpUpdate.update(client, mqtt_server,80,"/uploads/PTL_binary/PTL2.bin");//esp32
returns:
HTTP_UPDATE_FAILD Error (-7): HTTP error: no HTTP server

The only difference I have managed to spot between 2 .bin files are the groups:
See my picture below ( hopefully you can see
2020-09-30-084805_1920x1080_scrot.png

The original PTL.bin is owned by pi and has a group of pi

The PTL2.bin which comes from NAS, is owned by pi but has a group of root

Could that be whats causing the issues?
 

geekoftheweek

Joined Oct 6, 2013
1,201
Unfortunately when it comes to Windows and CIFS I don't have much to offer. I quit using Windows shortly after xp came out and haven't kept up with it. My guess is there is a setting somewhere that doesn't allow for the NAS, but no idea what to look fir
 

geekoftheweek

Joined Oct 6, 2013
1,201
I did a quick poking around the internet before heading out for the morning chores. It does seem that there are user / group name issues with network drives, but most of the questions I found were several years old and not exactly the same sort of problem. If you haven't already looked into it take a look at Samba. In ways the NAS would be better, but for just updating the ESPs it may be worth getting in to.

Maybe change the group and / or owner of the NAS directory to www-user or whatever Apache runs as and see if that helps.
 

geekoftheweek

Joined Oct 6, 2013
1,201
One more thought... have you checked your Apache logs? I don't know where they would be on the Pi (different Linux varieties seem to move them for whatever reason). There should be a couple lines in the httpd.conf that describes where they should be, and if you want to get carried away you can change what goes in them. If they aren't being produced you will need to change the group of the directory to whatever Apache runs as. They are somewhat cryptic at times, but usually contain enough info to point you in the right direction.
 
Top