Help in debugging Failure to Download ISO in Google Drive

[michaeldiaz]

Hello,

Can you help me where to start in debugging why are we cannot download iso using google drive? This is for Windows 10 Upgrade assistant.

Thanks,

[Cubert]

Please post images of your profile and of your google drive view in plugin.

Are files that you are using in plugin list?

[michaeldiaz]

Here is the profile:



Here is the google drive view in plugin:




Yes, the file are in plugin list.

Thanks

[Cubert]

Nice,

Now when you execute on an agent this policy it will log everything in script logs on agent that it is doing. What did it report when it started downloading the file?

Check agents script log for that entry to be present during the upgrade process time. Post that info back here.

[michaeldiaz]

You are referring to this right?



If so, here are the logs:

- Agents current OS version is -> 1809 - Policy version is -> 1909
- Making sure we have working directories on agent.
- Confirm we have enought working space on C:\ to complete upgrade.
- We are using Google Drive with API Key to download the ISO file.
- The ISO Download path is set to [https://www.googleapis.com/drive/v3/fil ... &alt=media]
- Downloading from URL [https://www.googleapis.com/drive/v3/fil ... &alt=media]
- The Windows 10 ISO failed to download the [%ltsvcdir%\Habitat\Windows10UG\1909.iso]. Please debug failure and retry script.

- The Script(6146) was successful in the Then section.

Script: S6146 - Starting at Server Time: Friday, July 17, 2020 12:51:52 AM
IF F5 ValueOne (P1): CMD Comparer (P2): 2 ValueTwo (P3): INSTALLNOW T: 0.8575214
L1 F139 Note (P1): Test for Windows OS on agent T: 0.8605098
L3 F139 Note (P1): 'Test for Master Switch to be on, we do not execute if switch is off or setting does not ex...' T: 0.8605098
L4 F20 SetType(P1):11 Parameter(P2):'SELECT count(Value) FROM `plug...' VariableName(P3):MasterSwitch T: 0.8605098
L5 F70 Value (P1): MasterSwitch Comparer (P2): 2 To (P3): 0 Jump (P4): :MASTERSWITCHISOFF T: 0.863527
L6 F139 Note (P1): See if we have a Policy ID passed to us T: 0.863527
L7 F70 Value (P1): POLICY Comparer (P2): 2 To (P3): Jump (P4): :WINDOWS10POLICYMISSING T: 0.864517
L8 F67 Sql (P1): UPDATE `plugin_p4a_habitat_windowsupgradepolicies` SET`LastUsed` = NOW() WHERE `ID` = '2'; T: 0.864517
L9 F139 Note (P1): Get Windows current version on agent T: 0.8655143
L10 F20 SetType(P1):0 Parameter(P2):'HKEY_LOCAL_MACHINE\Software\Mi...' VariableName(P3):Win10Version T: 0.8655143
L11 F70 Value (P1): Win10Version Comparer (P2): 8 To (P3): -9999 Jump (P4): :MISSINGWIN10VERSION T: 2.8763094
L12 F70 Value (P1): Win10Version Comparer (P2): 1 To (P3): Jump (P4): :MISSINGWIN10VERSION T: 2.8773178
L13 F70 Value (P1): Win10Version Comparer (P2): 2 To (P3): Jump (P4): :MISSINGWIN10VERSION T: 2.8773178
L14 F139 Note (P1): Load Upgrade Policy from SQL T: 2.8773178
L15 F196 SQL (P1): SELECT * FROM plugin_p4a_habitat_windowsupgradepolicies WHERE ID = '2' T: 2.8773178
L16 F197 Variable (P1): sqldataset RowIndex (P2): 1 T: 2.8893209
L17 F20 SetType(P1):1 Parameter(P2):1909 VariableName(P3):PolicyName T: 2.89532
L18 F20 SetType(P1):1 Parameter(P2): VariableName(P3):ISOWEBURL T: 2.89532
L19 F20 SetType(P1):1 Parameter(P2): VariableName(P3):ISO64BitFile T: 2.89532
L20 F20 SetType(P1):1 Parameter(P2): VariableName(P3):ISO32BitFile T: 2.8963148
L21 F20 SetType(P1):1 Parameter(P2): VariableName(P3):ISOENTFile T: 2.8963148
L22 F20 SetType(P1):1 Parameter(P2):1909 VariableName(P3):ISOVersion T: 2.8963148
L23 F20 SetType(P1):1 Parameter(P2):@sqlCopyFileLocally@ VariableName(P3):CopyFileLocally T: 2.8973145
L24 F20 SetType(P1):1 Parameter(P2):1 VariableName(P3):UseGoogleAPIKey T: 2.8973145
L25 F20 SetType(P1):1 Parameter(P2):0 VariableName(P3):UseNetworkCache T: 2.8973145
L26 F20 SetType(P1):1 Parameter(P2): VariableName(P3):NetworkCacheLocation T: 2.8983144
L27 F20 SetType(P1):1 Parameter(P2): VariableName(P3):NetworkUser T: 2.8983144
L28 F20 SetType(P1):1 Parameter(P2): VariableName(P3):NetworkPwd T: 2.8983144
L29 F20 SetType(P1):1 Parameter(P2):'https://pronto-core-cdn.pronto...' VariableName(P3):UserImageURL T: 2.8993185
L30 F20 SetType(P1):1 Parameter(P2):test VariableName(P3):UserMessage T: 2.8993185
L31 F20 SetType(P1):1 Parameter(P2):0 VariableName(P3):OverRideUserStopAction T: 2.8993185
L32 F20 SetType(P1):1 Parameter(P2):1 VariableName(P3):UseGoogleDrive T: 2.9003172
L33 F20 SetType(P1):1 Parameter(P2):'1HRelznoUIC0tjQNdtByxF5VD0aE89...' VariableName(P3):GoogleDrive32BitFileID T: 2.9003172
L34 F20 SetType(P1):1 Parameter(P2):'1kYmYATd2dSOLrxqIrjsd3B6mU6XV6...' VariableName(P3):GoogleDrive64BitFileID T: 2.9013274
L35 F20 SetType(P1):1 Parameter(P2):'1kYmYATd2dSOLrxqIrjsd3B6mU6XV6...' VariableName(P3):GoogleDriveENTFileID T: 2.9013274
L36 F20 SetType(P1):1 Parameter(P2):0 VariableName(P3):MasterCacheAgent T: 2.9013274
L37 F20 SetType(P1):0 Parameter(P2):'HKEY_LOCAL_MACHINE\SOFTWARE\Mi...' VariableName(P3):WINEdition T: 2.9013274
L38 F139 Note (P1): Test Policy ISO Version against Agent version T: 10.917237
L39 F70 Value (P1): Win10Version Comparer (P2): 8 To (P3): Value not found Jump (P4): :PREPAIRFORISODOWNLOAD T: 10.917237
L40 F29 Message (P1): Agents current OS version is -> 1809 - Policy version is -> 1909 T: 10.917237
L41 F70 Value (P1): Win10Version Comparer (P2): 7 To (P3): 1909 Jump (P4): :WINDOWSUPTOVERSION T: 10.9202374
L42 F70 Value (P1): OverRideUserStopAction Comparer (P2): 2 To (P3): 1 Jump (P4): :GETUSERINPUT T: 10.9212591
L43 F139 Note (P1): Prepair for ISO file download T: 10.9212591
L44 F139 Note (P1): :PREPAIRFORISODOWNLOAD T: 10.9212591
L45 F29 Message (P1): Making sure we have working directories on agent. T: 10.9212591
L46 F160 Path (P1): %ltsvcdir%\Habitat\windows10UG\ T: 10.9232387
L47 F154 Folder (P1): %ltsvcdir%\Habitat\windows10UG\ T: 14.9332762
L48 F29 Message (P1): Confirm we have enought working space on C:\ to complete upgrade. T: 31.0659318
L49 F235 T: 31.06893
L50 F70 Value (P1): CDRIVESIZE Comparer (P2): 4 To (P3): 15 Jump (P4): :NOTENOUGHTDRIVESPACE T: 41.0920715
L51 F70 Value (P1): UseGoogleDrive Comparer (P2): 2 To (P3): 1 Jump (P4): :GOOGLEDRIVE T: 41.0920715
L61 F139 Note (P1): :GOOGLEDRIVE T: 41.0920715
L62 F70 Value (P1): UseGoogleAPIKey Comparer (P2): 2 To (P3): 0 Jump (P4): :GOOGLENOAPIKEY T: 41.0930652
L63 F29 Message (P1): We are using Google Drive with API Key to download the ISO file. T: 41.0930652
L64 F20 SetType(P1):11 Parameter(P2):'SELECT `Value` FROM plugin_p4a...' VariableName(P3):Google_API_Key T: 41.0950677
L65 F70 Value (P1): Google_API_Key Comparer (P2): 2 To (P3): -9999 Jump (P4): :NOGOOGLEAPIKEY T: 41.0980682
L66 F70 Value (P1): Google_API_Key Comparer (P2): 2 To (P3): Jump (P4): :NOGOOGLEAPIKEY T: 41.0990709
L67 F70 Value (P1): WINEdition Comparer (P2): 8 To (P3): Enterprise Jump (P4): :GOOGLEURLENT T: 41.0990709
L69 F20 SetType(P1):1 Parameter(P2):'https://www.googleapis.com/dri...' VariableName(P3):ISODOWNLOADPATH T: 41.1000727
L70 F129 Jump (P1): :CONTINUETOINSTALL T: 41.1000727
L89 F139 Note (P1): :CONTINUETOINSTALL T: 41.1000727
L90 F70 Value (P1): ISODOWNLOADPATH Comparer (P2): 2 To (P3): Jump (P4): :MISSINGISOFILENAME T: 41.1010706
L91 F70 Value (P1): ISODOWNLOADPATH Comparer (P2): 2 To (P3): \ Jump (P4): :MISSINGISOFILENAME T: 41.1020712
L92 F29 Message (P1): 'The ISO Download path is set to [https://www.googleapis.com/drive/v3/fil ... ATd2dSOLrx...' T: 41.1020712
L93 F139 Note (P1): Do we check for Network Cache? T: 41.104067
L94 F139 Note (P1): Download ISO and Extract to working directory T: 41.1050709
L95 F20 SetType(P1):1 Parameter(P2): VariableName(P3):ISOPATH T: 41.1050709
L96 F70 Value (P1): UseGoogleAPIKey Comparer (P2): 2 To (P3): 1 Jump (P4): :DOWNLOADUSINGGOOGLEAPIKEY T: 41.1050709
L166 F139 Note (P1): :DOWNLOADUSINGGOOGLEAPIKEY T: 41.1060727
L167 F70 Value (P1): UseNetworkCache Comparer (P2): 2 To (P3): 1 Jump (P4): :DOWNLOADTONETWORKCACHE T: 41.1060727
L168 F29 Message (P1): 'Downloading from URL [https://www.googleapis.com/drive/v3/fil ... Irjsd3B6mU...' T: 41.1060727
L169 F70 Value (P1): WINEdition Comparer (P2): 8 To (P3): Enterprise Jump (P4): :VERIFYNETCACHEENT1 T: 41.1160707
L171 F19 URL (P1): 'https://www.googleapis.com/dri...' LocalFile (P2): '%ltsvcdir%\Habitat\Windows10UG...' Wait (P3): 0 T: 41.1170713
L173 F114 File (P1): '%ltsvcdir%\Habitat\Windows10UG...' Check (P2): 1 Jump (P3): :WINDOWS10CACHEFAILED T: 51.1312094
L333 F139 Note (P1): :WINDOWS10CACHEFAILED T: 53.1451748
L334 F29 Message (P1): 'The Windows 10 ISO failed to download the [%ltsvcdir%\Habitat\Windows10UG\1909.iso]. Pleas...' T: 53.1451748
L335 F67 Sql (P1): 'INSERT IGNORE INTO plugin_p4a_habitat_win10upgrades_agents (`ComputerID`,`AttemptLog`) Val...' T: 53.1481664
L336 F129 Jump (P1): T: 53.1511852

[Cubert]

Awesome,

Ok so here out of all that log quagmire is the skinny.

L168 F29 Message (P1): 'Downloading from URL [https://www.googleapis.com/drive/v3/fil ... Irjsd3B6mU...' T: 41.1060727
L169 F70 Value (P1): WINEdition Comparer (P2): 8 To (P3): Enterprise Jump (P4): :VERIFYNETCACHEENT1 T: 41.1160707
L171 F19 URL (P1): 'https://www.googleapis.com/dri...' LocalFile (P2): '%ltsvcdir%\Habitat\Windows10UG...' Wait (P3): 0 T: 41.1170713
L173 F114 File (P1): '%ltsvcdir%\Habitat\Windows10UG...' Check (P2): 1 Jump (P3): :WINDOWS10CACHEFAILED T: 51.1312094
L333 F139 Note (P1): :WINDOWS10CACHEFAILED T: 53.1451748
L334 F29 Message (P1): 'The Windows 10 ISO failed to download the [%ltsvcdir%\Habitat\Windows10UG\1909.iso]. Pleas...' T: 53.1451748

This part of the log is telling you that it launched the script function "Download From URL" with the option to wait for completion. After that it called script function to see if file existed and if not goto ":WINDOWS10CACHEFAILED" which logs the failure and exits script.

So the scripting is doing what it should be doing but something either with the URL limits or the agents ability to write a file to the destination directory of ISO will cause this issue.

2 considerations are:

Is google limiting the number of downloads per day (They do this)? If so use the network cache settings to store a copy locally to network for all other agents to use instead of each downloading the ISO separately. This will allow every agent at a location to get the ISO with out limits and reduces bandwidth needed. Schedule 1 agent to run a day ahead of all others so ISO gets populated on the local network before other agents start their updates and you should not have an problems.

The second is local permissions, is agent service able to write file to the directory ? During an update, monitor the ISO download directory to see if you see the ISO starting to save during download. You should see byte count rising on the filename as download occurs. Does it every make it? if not its permissions if file does show up then it maybe download stalling due to google or bandwidth need what interrupts downloads. But then I would think that the test file would pass as file name exists at this point, which I believe is not happening.

[Cubert]

Also the time between download (T: 41.1170713) and download test (T: 51.1312094) was ten seconds so this failed right at the start of the download process. which would be a rejection from google or permissions for saved file location.

[jallenEITP]

We are having the same issue. We did not have the issue initially but now it is failing.

[Cubert]

Ok so you either have reached quota or do not have the file permissions on google file set to allow it to be publiclly downloaded.


Fin in your logs the following log line.

- Downloading from URL [https://www.googleapis.com/drive/v3/fil ... &alt=media]

Click the link in the log to see what google spits back.

[jallenEITP]

Looking at the client the script ran on it shows this file downloaded.
09/01/2020 11:55 AM 9,728 Habitat-GDLFD.exe

but no ISO

I can download the file by the URL manually

The Habitat-GDLFD.exe file is not listed in any version of the bit version for 32 / 64 or Enterprise