Help needed to repair Ford Travelpilot NX

Discussion in 'Automotive Electronics' started by picoamp, Sep 13, 2018.

  1. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    Hello! I'm new to this forum and joined in the hope to find people like me, who are interessted in repairing things and let other participate.
    Maybe this is the totally wrong audience, so please be kind and tell me.
    I'm highly interessted in any automotive electronics and also done some research and projects in this area. I also run a (German) website mk4-wiki.denkdose.de where i try to publish some of my solutions and backgrounds to it. I'm also a member in other electronics-communities but none of them is really handling this.

    One of my current projects is to find out what causes the Blaupunkt (Bosch) "Travelpilot NX", which is build into many Ford vehicles, to shutdown after powering on. It just keeps running for some seconds and all seem ok, but then it shut off the display. After this, i can be pushed on two more times, where it does the same. Then it keeps off until power is pulled and put back again.

    I've done serious research into that issue and also some repair tests. The system has two main components connected together, the mainboard and the graphicsboard. They can operate indepandently, so it is possible to pull the graphicsboard (with the touchscreen and buttons on it) and the mainboard will boot properly. Of course it is of no sense, because without the HMI one is unable to do anything with it. But it would communicate over CAN and would also play music of this is done while the Satnav is running. Funny, but very interesting for testing purposes!

    So i found out the problem resides on the mainboard, because i swapped the graphicsboard to a working unit where it also works fine.
    I can see on the current consumption of my switching power supply how it behaves. Also on the mainboard there is a HMI processor (OMAP5948) controlling all the funny stuff and a radio processor. They are coupled by an SPI interface or such. Also, those two components work more or less independant. So the HMI restarts when the display goes off, but the radio proc keeps running. This makes me pretty shure that the problem must be somewhere around the HMI proc. Maybe some sort of timeout causing the self reboot.

    I've changed the RAM-Chips to be shure they are not faulty and causing this. Also changed the serial EEPROM, but this does'nt change anything either.
    Next i tried to measure the power regulators, there are many of them for the various chips on the board. But the seem (!) okay.

    So, if anyone is also into this topic and want to exchange his knowledge with me, be welcome! :)


    THX
     
    Last edited by a moderator: Sep 13, 2018
  2. wayneh

    Expert

    Sep 9, 2010
    14,691
    5,211
    Board-level repair of such a complex device will be very hard without a service manual and schematic information. You need test points, testing protocols and so on. It could be something very trivial to repair, but nearly impossible to diagnose. Have you researched what a mainboard replacement might cost? I mean, there's probably a few in junk yards looking for a new home.
     
  3. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    Thanks, wayneh, but maybe you misunderstand my targets. It's not only to bring the device up running, but to learn more about it.
    I have the schematics of it, but no further information. Unfortunately the schematics are under NDA so i can't post it here. And also i have a working unit by hand, to compare. But there are hundrets of MPs on it.
    You are right about the complexity of the system, but i thing it's more a logic-puzzle. What can go wrong to cause the HMI restart itself?
    What i am looking for is some people who may be in the same situation like me, and wants to share their knowledge about it.
    Just for example, for the Travelpilot FX i found out that faulty RAM-chips causing the unit to not boot after poweron. After finding the genuine one as spare parts, i was able to fix this issue. Also described in my Wiki linked above.
    So, it does not scare me if it is complex :)
     
  4. wayneh

    Expert

    Sep 9, 2010
    14,691
    5,211
    Just a wild guess based on some personal experience with a TV circuit: The processor may look for a number of inputs from other parts during it's start-up routine and, if it sees something askew, it shuts down and tries again. Power supply voltages might be among those input checks but it might also look for a response from a sub-system. If you can identify the pins it is checking, you might be able to follow the startup process on a working unit and see if you can find something out of line.
     
  5. bwilliams60

    Well-Known Member

    Nov 18, 2012
    948
    151
    Welcome to AAC. Just a quick background from my experience on AAC, there are many great people on here willing to bounce things off of and every once in a while you will find someone who has an intelligent answer to your question. This was an electronjcs forum long before it was opened to automotive so it is slowly coming around. It is nice to see more car guys coming because the electronics on vehicles is getting larger by the minute. I enjoy ECM, PCM work and although I dont fully understand the electronics side of everything, I seem to be able to comprehend most of the stuff that goes on in these modules. Then I ask how these two components should work together and the electronics guys will jump in and fill in the holes. Pretty good bunch here.
    As for your problem, have you disconnected the battery negative cable from the battery and then shorted it to the positive cable to form a hard reboot. Second question is, have you reprogrammed the TravelPilot with the latest greatest software. In Canada we have solved a lot of audio programs by reflashing the unit. Just a thought.
     
    wayneh likes this.
  6. wayneh

    Expert

    Sep 9, 2010
    14,691
    5,211
    Excellent suggestion, if there is such a protocol for this particular device. Always rule out software before you go after the hardware.
     
    bwilliams60 likes this.
  7. jpanhalt

    Expert

    Jan 18, 2008
    6,279
    1,166
    That seems to be a common problem. It's a Ford.

    First few hits on Google revealed:
    https://www.ecoustics.com/electronics/forum/car-audio/736343.html
    https://www.justanswer.co.uk/ford/7qjzv-factory-fit-touch-screen-sat-nav-s-max-2008-turns-off.html
    Apparently, there is a software update:
    https://www.techwalla.com/articles/how-to-reset-a-blaupunkt-travelpilot

    Have you done the software updates?
     
    bwilliams60 likes this.
  8. bwilliams60

    Well-Known Member

    Nov 18, 2012
    948
    151
    Good work jpanhalt. Will add this to my library. I was just going on experience with other Ford radio problems.
     
  9. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    Hi Folks!
    First of all, i would thank you all and ensure i really appreciate your help!! :)
    But the links above addresses a totally different problem, expect one, which was my own crosspost in another community ;-) Forgive me, but as i told in my introdution, i look for helpers everywhere

    The shutdown issue occurs without any software change. And why should it run with an arbitrary version for years and suddenly don't? This makes no sense to me. No, it's like in the computerworld, answer the question "what has changed?", only if things change, they would behave unexpectedly. Nontheless is it not possible to upload any firmware while the issue is there. To start an firmware-update one must insert da special CD into the DVD-drive where usually the navigationdata reside. But there is no time to got there because the unit will shutdown earlier than you can enter the PIN code. Well, there is a way of course, if you use JTAG. With my equipment i was able to download the whole mainboard flash via JTAG, but can't upload/program it. Therefore i need different licence which is too expensive for me (about $1.000).
    I already downloaded the firmware from both devices, but as they are not exactly the same builddate they slightly differ which may be ok.

    So, let's focus on the hardware. As i said the issue must reside on the mainboard. I have a working one and the one with the issue here at hand. So i can compare anything i want. Also i could swap some parts, expect of BGA ones which i can't resolder. Unfortunately the mainboard-flash IS BGA :-( so it is not possible for me to swap them over to ensure the content is noch corrupted somewhere. Yes, there could be a little error in the interrupt-vector-table and the system crash. So just some bits could be wrong to produce this issue.

    But how can i write the flash using JTAG without using J-Flash (which is limited to reading in the free version)?

    I've also had an Olimex JTAG based on FTDI-Chip which will run with OpenOCD, but i did not manage to connect with it to the target. Something is missing in the initilization procedure, so the OMAP5948 could not be halted, which is neccessary to dump the flash connected to it. This is another issue i get stuck... :-(

    I'm not sooo familiar with JTAG, but if one, it could be a way to find out what is going on.
     
  10. bwilliams60

    Well-Known Member

    Nov 18, 2012
    948
    151
    So if we want to approach this from a hardware point of view, you say you have a schematic, can you share it on here?
    If you can, lets follow th power supply and see where we go with it. Shouldnt be too hard to see what is shutting down.
    Is it possible this is temperature related?
     
  11. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    Hi bwilliams60! Thanks for you support. Yes, i'm in charge of the whole schematics, but they are "non-distributable" (NDA), so if i post them here, i would risk violating some laws, which i does not want, even if the device is about 10 years old.
    I've checked temperature sensitivity already without any result (which does not mean that there aren't any...).

    Thinking in modules or blocks, there is a block with "power supply" and one with "power control". First one keeps all the voltage regulators for those levels: 1.6V, 2.5V, 3.3V, 5.0V, 7.5V, 8.5V (there are more than one regulator for each named level). The power control unit consists of switches and measurepoints to check if some voltages are below the needed value and the switches powers on/off parts of the system, needed for powersafe modes and such.

    First i try to mease all voltages from the regulators i found, and think they are ok. Also the low-voltage-pins seems to have the right level. Something else is causing the HMI to restart while voltages are all ok and stable. Unfortunately my testing capability is limited to a Rigol 4 channel DSO 1054Z and a simple 8 channel Logicanalyzer. I guess i had to check all those signals in parallel to find, which causes what.

    I wish i could share those information with someone from here, but i fear it is against the rules of this forum to post company properties.

    P.S.: How to start a private conversation?
     
  12. wayneh

    Expert

    Sep 9, 2010
    14,691
    5,211
    It's a very rare contract that would extend to this length of time. Depending on the state, contracts that extend to perpetuity are not even enforceable - ie. they're worthless. A good contract needs a termination date or at least a renegotiation date.

    All I'm saying is that you may not be constrained any longer. You could read the details and perhaps you'd be free to do as you wish.
     
  13. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    wayneh, don't get me wrong. I have no problem with sharing, especially to get help, but regardless of any contract, it's company proberty of Bosch and they won't be happy with publishing internal data anyhow.

    Yesterday i went through the power supply/control schematics again. I could measure all voltage levels given there. Every voltage regulator seems to output the correct level and working fine. The voltages are there all the time, until the unit totally shuts off.

    Let me explain some things i found out until now:

    Poweron
    After applying the 14V on the AK Interface (Quadlock), the LTC3827 dual-power regulator gets started, producing +5V and +3.3V and the Radio-processor (RU) starts up. After this the RU enables all the other voltage regulators and thus starting the rest of the system, e.g. the HMI processor.
    Besides an SPI-Bus, the RU uses those signals to control the HMI section:
    TUN_CTRL_IF:RU_RST_HMI -> goes from RU to HMI and could be interpreted like "Hey, HMI restart yourself"
    TUN_CTRL_IF:HMI_RUN -> comes from HMI to RU and is HIGH if the HMI is running (even at bootup)
    TUN_CTRL_IF:HMI_PWR_ON -> goes from RU to the power-control section and enables the Mosfets to switch power for the HMI section (RAM, Flash, OMAP) and therefore is able to start and shutoff the HMI.

    Current draw "footprint"
    Without the frontpanel, the systems currentdraw goes up to about 600 mA within 3 seconds. After that the system is fully bootet and responsive. Normally it would stay there (a little more or less) but on the faulty board the current falls down to 125 mA after 12 seconds from poweron. Sometimes i see an additional drop after 6 seconds or so. Maybe my powersupply LED-display is to slow to show it anytime. Within the 12 seconds, the HMI has restartet 3 times (if the frontpanel is connected and the radio is "tipped" on). After another 6 seconds from here the current goes down to 10 mA, which is like a sleep mode. Then, after 60 additional seconds it goes to 0 mA (deep sleep).
    This process is repeatable at any time and does not change if the PCB was heated up (+40°C) or cooled down (-10°C) before. So i seems there is no temperature sensitvity here.

    With the frontpanel applied, the timing and powerdraw is a bit different. Shure, it draws more current, but after tipping the ON button, the display apprears with the PIN-code message and you could type (touch) in and maybe if you are fast enought can see the main screen of the radio before the display is gone again. Then it needs about 2-3 seconds before it processes the ON button again, starting with the PIN. It looks like it the HMI has rebootet. After three times doing so, the system does not start again after tipping ON. In whole, this lasts longer as without the frontpanel.

    Bootstrapping
    The Radio-processor boots first, from its internal Flash memory. It then enables all the other voltage regulators providing power to the subsystems. This also starts the HMI-processor, which starts with an bootloader from it's internal Flash. This bootloader transfers all the contents of the external Flash-Memory (Spansion, BGA) to an also external RAM (two Micron SDRAM Chips). After that, the HMI starts executing the software in RAM.

    Watchdog
    Besides of checking the voltage levels, i know that the Radio-processor works like a watchdog for the HMI-processor. If i "HALT" the CPU of the OMAP5948 (HMI) by JTAG, the system shuts down after about 12 seconds. This is because the RU (Radioprocessor-Unit) is not getting any heartbeats from the HMI (ARION) and think it's crashed. Then it shuts down the whole radio until power is reapplied. This was the first issue i encounter when trying to read all the contents from the Mainboard-Flash via JTAG. I then found out that the RU uses an input-signal called DNL_PWR_ON, if set to HIGH, will prevent from shutting down. It somewhat disable this watchdog. So it was possible for me to download the whole content of the flash in any time it needs :) The RU and the HMI shares an SPI-Bus to talk to eachother. I guess the heartbeats goes here.


    Conclusions so far:
    After measuring all again i'm pretty shure all the voltage regulators are working. I also measure the switched voltages, which are controlled by the Radio processor to enable other parts like the GPS-receiver, FM-Tuner, HMI and so on. All voltages are there, all stable over the whole process described above. I also checked the voltage-drop signals, which seems also fine and equal on both systems. This is also true for the various reset-signals, produced by voltage-watchdog-chips like MAX6421, MAX6389, MAX6776, etc.

    As tell above the RU works like the central controlling unit for the whole system. I can't find any hardware problem which may causing this. The timing is very exact. I guess without the frontpanel, the system was not poweron and so the RU watchdog shutdown the HMI after 12 seconds. If the panel is applied, the poweron-Button causes the HMI so send heartbeats, but after some seconds something inside the HMI crashes and cause an direct reboot. The HMI-processor (OMAP5948) does have an internal watchdog also (Coprocessor), so it is very likely that it is able to restart itself without the RU to interfere. After three crashes it may give up and therefor does not start again by pushing the power button. Now the RU comes into play, because it does not receive any heartbeats and goes into powersafe mode.

    Well, i guess we are faced with an softwareproblem here. Something running as a timer or background-task is crashing and forces a reboot. Another part of the HMI limits this to three retries before it gave up. Now, i wonder if there is a way to debug the software, which i could download from the mainboard-flash. Unfortunately i could not upload (flash) any content with my Segger J-Link EDU, because i do not have a license for J-Flash to do so. And for my alternative JTAG-Flasher tool, Olimex ARM-USB-OCD-H, i did not manage to download at all. I could connect, but not HALT the CPU. The OMAP5948 is not publicity available and so i have no clue what J-Flash does to make it halt. It would be great to transfer the software from one device to another to see if the error moves with it.
    The second option was to swap the flash chips. But because their are BGA it will be diffucult for me to do. I have no problem soldering TSOP but BGA. Desoldering is easy, which i did on an testboard, but i did not manage to resolder it firmly that it will work afterwards.


    Oh, btw. nice cat's - yours? ;-) Mine is like the left one.
     
  14. wayneh

    Expert

    Sep 9, 2010
    14,691
    5,211
    Yup, they're mine. I lost the one on the left this summer. He was 17 or 18 and showing his age, but met a sudden end when the wife ran over him. Thank god I wasn't home at the time. Best cat I ever had.
     
  15. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    After taking a walk, there one thing coming in my mind. The radioprocessor needs more attention. Maybe someone knows more about this processor type used? It is an µPD 70F3283 (YGD) from NEC and datasheets are available for public. So there should be plenty of information and knowledge around. Also these kind of µCs are used in many automotive applications. The series is called V850/SG2.

    I wonder if it is possible to get more information from it. Maybe via the debug-interface. There are some signals i found on the debug-plug of the mainboard.
    /V850_RESET
    Goes to Pin 14 of the V850 directly. This port is named /RESET
    /POR_RESET is also connected here via a reverse polarised (blocking) shottky diode, so if /POR_RESET (Power-on-reset) goes low, the /RESET on the V850 goes low also.

    V850_MON_IN
    Called "MBS_SCI_RXD" is going via an 100 Ohm resistor to P31/RXDA0/INTP7/SIB4 (Pin 26).

    V850_MON_OUT
    Called "MBS_SCI_TXD" going through two 100 Ohm to P30/TXDA0/SOB4 (Pin 25).

    V850_VPP_ON
    Goes through a leading shottky and an resistor to PDH5/A21 (Pin 7) as "DNL_VPP_ON" and also to IC/FLMD0 (Pin 8) as "DNL_VPP_IN".
    I guess there are for programming the chip only.

    V850_MUX
    This special signal goes via an 47k resistor to the internal signal "TUN_CTRL_IF:HMI_RST_RU". A low on this port will block HMI_RST_RU to occure.

    I wonder how to get debug messages out here. The TX and RX lines seems passive. Maybe one need to emitt some data here to make the V850 talk. Do someone have any further information? I bet this is a standard debug wire port used here...
     
  16. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    Besides of looking for a way to get any debugging output messages, i was looking for an alternate way to reprogramm the mainboard-flash. I tested this signal i found above, „V850_MUX“ (set to GND) and it prevents the RU from going into powersafe-mode after the HMI has restartet itselft the third time. So now, it loops forever, with a tiny extra pause between each three tries. I wonder what this signal was intend to do...

    This brings me to the idea to test a bit with this new situation. I was able to enter the PIN and press „OK“ within the 3-4 seconds where the radio remains on until the HMI reboots. After that, the next start with the powerknob does not request the PIN-Code anymore, of course only until i powercycle the device. I wonder how the devices distinguishes between those two states. I guess the HMI is only in charge of displaying and verifying the PIN entered, because it gets restartet, and the RU keeps track of the current state.

    As you might not know, there is an Update-Procedure for the system. One must insert a CD with an special folderstructure into the DVD-drive, where normally the navigationdata disc resides.

    My plan was to check if i am able to let the system access this disc and maybe start the update procedure. Therefore i used my working board to eject the navdata DVD from the drive and inserted the update CD. Unfortunately the remaining on-time is not sufficient to go into the submenu where to start the update, nor does it detect the update itself automatically after poweron.

    There must be so e other ways to do an update from external storage because there are signals named USB_DNL and EXT_BOOT on the serviceport. It would be great to find a way to let the OMAP load and boot from another storage than the internal Flash. This may be a way to get around that softwarecrash.

    Any ideas here?
     
  17. bwilliams60

    Well-Known Member

    Nov 18, 2012
    948
    151
    Just curious but did you do the hard reset on this by disconnecting the battery and shorting the cables together? Let it sit for 10 minutes disconnected and then reconnect and try. Sounds too simple but most of the Ford stuff we have run into is cured by this or a reflash from Ford "As Built".
    What you are speaking of in your posts are way beyond what most go through on here so I am not sure where you will find help on this. Without seeing diagrams on the circuitry, we are all just guessing.
     
  18. picoamp

    Thread Starter New Member

    Sep 13, 2018
    10
    0
    Hey bwilliams60, could you get in contact with me by mail? For some reason i am not aware (or able) to start an private conversation here.
     
Loading...