Habitat 3rd party application manager not working

Support forums for the Habitat Automate plugin
User avatar
Cubert
Posts: 2432
Joined: Tue Dec 29, 2015 7:57 pm
8
Contact:

Re: Habitat 3rd party application manager not working

Post by Cubert »

So you got an OK returned..

When you get an OK from a Automate command as a return, then most likely something went south during execution and instead of returning a error it "crashed" and Automate returns a OK...


Otherwise you would get either a POSH error returned or a "True or False" as the last POSH command on the script is to test the file that was copied to the share. True if file exists and false if it does not. OK does not have any relevance in the code, we do not return any "OK" as part of that script.

So...
Now we need to test the script first hand to make sure POSH does not explode when the script is executed.

On that agent, remote into desktop and open as an "run as Administrator" the PowerShell ISE and copy the script code as you have it now into the ISE.

Execute it and see what is returned. I am sure it either will result in an error or it will complete as expected.

If it completes as expected then the share is good and working as expected, we have an agent script to agent POSH issue. We should then try some basic POSH commands on the agent through the plugin to see if there is more with POSH inside the agent. The POSH engine that the script uses may not be the same that the ISE uses and more testing of agent POSH may show other issues.

If it fails then the POSH should revile the error and the line in the code it fails at. We can then see why it does not like what it sees.

Just a side note, Since this seems to be a uncommon issue as we have many MSPs using the Habitat App manager and the caching it produces without the issue we are seeing here. Typically it shouldn't be this hard to get rolling if your Network Share permissions are good. Some general and simple thing to try to do to narrow down where your issue lies.

To rule out agent specific issues:
  • How many clients have you deployed caching to thus far?
  • Are all the locations and clients having this issue or is this a one off client/location?
  • Have we tried a different agent?
  • Have we tried an agent that has POSH 5.1 installed currently?
post back your findings.

eurgello
Posts: 20
Joined: Thu May 26, 2022 4:02 pm
1

Re: Habitat 3rd party application manager not working

Post by eurgello »

Hi,

Apologies for the late response on this.

I tested out that script locally on the server via Powershell ISE and it was able to complete. Packages has been copied to the cache folder.

Answers to your questions are below:

How many clients have you deployed caching to thus far? We should have around 5-10 clients that we have setup the cache share folder. None have we setup for them to have a caching agent so should be getting the packages directly from chocolatey.
Are all the locations and clients having this issue or is this a one off client/location? All of them are having the issue.
Have we tried a different agent? Not sure if this is applicable. I can try and setup a caching agent for one of our clients to try this out.
Have we tried an agent that has POSH 5.1 installed currently? Currently the server I set as the caching agent is using ver 5.1.

User avatar
Cubert
Posts: 2432
Joined: Tue Dec 29, 2015 7:57 pm
8
Contact:

Re: Habitat 3rd party application manager not working

Post by Cubert »

I tested out that script locally on the server via Powershell ISE and it was able to complete. Packages has been copied to the cache folder.
Ok so share is set with correct permissions and agent can make the copy using native POSH.


We should have around 5-10 clients that we have setup the cache share folder. None have we setup for them to have a caching agent so should be getting the packages directly from chocolatey.

I'm a bit confused here. You have client locations setup with caching shares but no assigned caching agents for those locations? They are all set to use direct Chocolatey access?

Ok so all agent at the locations in question all have the (#) symbol in front of the agents name? This is the standalone client setting. If you have all agents at a location set this way and there is more than one or two agents there then your going to have some issues getting anything done for that location.

every location using caching must have a caching agent unless there are VPN connected network shares. Only then can you have one agent to manage the share cache for all locations able to access that share.

You can switch caching agents at any time and the caching agent does not need to be a server. It can be but not required.

If one agent is having caching issues then try switching to another agent and see if it has better results.


From the current troubleshooting it feels like the current agent has some POSH issue with the internal POSH the script engine is using. Try a different agent to see if results are the same.

eurgello
Posts: 20
Joined: Thu May 26, 2022 4:02 pm
1

Re: Habitat 3rd party application manager not working

Post by eurgello »

I'm a bit confused here. You have client locations setup with caching shares but no assigned caching agents for those locations? They are all set to use direct Chocolatey access?
Yes, they are all set to use direct Chocolatey access.

They don't have the # symbol though. They are configurated as normal agents and not caching or standalone agents. Wouldn't normal agents still get the packages directly from Chocolatey if there isn't a caching agent configured?

I just tried it on 2 different agents (1 server and the other is a WS) and both have the same result, OK.

User avatar
Cubert
Posts: 2432
Joined: Tue Dec 29, 2015 7:57 pm
8
Contact:

Re: Habitat 3rd party application manager not working

Post by Cubert »

Ok my confusion was correct. I believe you maybe missing how each agent type interoperates with the plugin. Let me try and explain again how the plugin handles deploying applications via automation.

There are only 2 reliable scenarios that will produce the desired results of deployments and updates in a timely manner.


Either (Caching agent-normal agent as a group) or (standalone)

If an agent is set to Standalone, (#) will appear in front of agent name and it will then and only then start attempting to use the public repositories directly from the chocolatey.org site. The key word here is "attempt". The reason for this is that to many standalone agents at one physical (IP address) location will cause the public repo to start with throttling and if the requests get high enough the repo will start blocking the requests for a given amount of time. This will cause agents to go with out deployments and updates as only the first 2 agents will ever get access to repo from that IP address.

This is where the (Caching agent - normal agent group) fixes all the problems that the standalone causes. By setting 1 agent responsible for getting software and all normal agents only getting software from cache resolves the issues of over burdening the repository from any one IP address (Location). The Key here is that Normal agents do not go directly to repo but only go where told. So if you do not have 1 agent at that location set as the caching agent then the system will not work. There is no agent responsible for delivering the packages to cache share so all normal agents see zero packages available..

Standalone is designed for that home worker or single laptop that could end up at any IP address at anytime. Not really meant to use in mass at any one location unless the location was just a logical group of remote individual locations like home workers.


So my confusion was why have a share and normal agents without a caching agent?

Without the caching agent for the location the system is not completely setup yet and so no action will take place for location. There maybe times that a caching agent has an issue for some reason or another. In this case it is easy to switch the caching agent duties to any other agent at location. Just select agent and set it to caching. it will cause the current caching agent to return to a normal agent. No other configurations are needed as it will use the same creds as the network share is already set for.

I hope this better clarifies how the 2 types of setups function.

The other option is to purchase a license for each agent you want to run standalone, In the plugin we provide a license importer that will take the XML license code they give you and transfers that to all agents to pre license them so that they do not trigger the throttles and blocks that multiple unlicensed agents will do from a single location. Then you can forgo any caching in this case.

Here is that fee per agent. If you use caching for a location this is not required, only if you want to run a location as standalone that has lots of agents and you want those agents to actually complete the installs and updates without blocks and bands.


Screenshot 2022-07-14 092807.png
Screenshot 2022-07-14 092807.png (67.69 KiB) Viewed 5352 times

eurgello
Posts: 20
Joined: Thu May 26, 2022 4:02 pm
1

Re: Habitat 3rd party application manager not working

Post by eurgello »

Thanks for the clarification on this. The reason we had the share folder setup was for other plugins and updates that weren't managed by Habitat.

I'll reach out to my manager then to setup the caching agent for each client.

But how about the issue with the packages still not being updated on the share folder even though caching agent has been setup? I tried that power shell script on 2 other agents and set them up as a caching agent at the time and still having the same result.

User avatar
Cubert
Posts: 2432
Joined: Tue Dec 29, 2015 7:57 pm
8
Contact:

Re: Habitat 3rd party application manager not working

Post by Cubert »

Since the same command is run from the script I suspect its something agent related. I would attempt another agent before spending a lot of time troubleshooting. If multiple locations, multiple clients all start having the same issue it may be a bigger issue. Lets see if it resides to either this network or the current caching agent.


Lets change caching agents at this location and see if the results are the same?

eurgello
Posts: 20
Joined: Thu May 26, 2022 4:02 pm
1

Re: Habitat 3rd party application manager not working

Post by eurgello »

Hi,

Apologies for the late response on this. We had to rebuild a computer to test this out as we didn't have any other domain computers available. So this a newly imaged domain computer. I have added the cache folder, shared it out and configured it in Automate. Afterwards made it as the caching agent.

Unfortunately issue is still the same. Cache folder doesn't get the app packages. I do see the app packages under the Habitat folder in Windows. I ran the Habitat powershell command and response is still OK.

Any logs that we can provide to help ascertain the cause of this?


Thanks!

User avatar
Cubert
Posts: 2432
Joined: Tue Dec 29, 2015 7:57 pm
8
Contact:

Re: Habitat 3rd party application manager not working

Post by Cubert »

Then I would start back here with this post. From the sounds of it you are getting the packages downloaded but the NET USE part of the POSH script is failing so the copy then fails and the test to see if files moved never runs. thus you do not get a True or False back from script.

Execute the script manually on agent and see what it errors out as. Typically at this point it is either a "path" issue or a permissions issue. These are the 2 variables your passing down to NET USE. The network path and the username and password.

Is this a domain computer? if may have policies set preventing access, maybe firewall on agent or the creds are not passed correctly for the domain type. There can be a lot of different reasons so I would run Automate script in debug mode with couple of breaks around the POSH script execution and would test execute the POSH direcly on agent to see where its going south.


The actual script POSH script that is doing the copy is a very simple process so very little to control in the script. It downloads the files locally then does the NET USE, Copy *.* and TEST in just 3 lines. Failures tend to be at the share level.


Cubert wrote: Thu Jul 07, 2022 1:19 pm So you got an OK returned..

When you get an OK from a Automate command as a return, then most likely something went south during execution and instead of returning a error it "crashed" and Automate returns a OK...


Otherwise you would get either a POSH error returned or a "True or False" as the last POSH command on the script is to test the file that was copied to the share. True if file exists and false if it does not. OK does not have any relevance in the code, we do not return any "OK" as part of that script.

So...
Now we need to test the script first hand to make sure POSH does not explode when the script is executed.

On that agent, remote into desktop and open as an "run as Administrator" the PowerShell ISE and copy the script code as you have it now into the ISE.

Execute it and see what is returned. I am sure it either will result in an error or it will complete as expected.

If it completes as expected then the share is good and working as expected, we have an agent script to agent POSH issue. We should then try some basic POSH commands on the agent through the plugin to see if there is more with POSH inside the agent. The POSH engine that the script uses may not be the same that the ISE uses and more testing of agent POSH may show other issues.

If it fails then the POSH should revile the error and the line in the code it fails at. We can then see why it does not like what it sees.

Just a side note, Since this seems to be a uncommon issue as we have many MSPs using the Habitat App manager and the caching it produces without the issue we are seeing here. Typically it shouldn't be this hard to get rolling if your Network Share permissions are good. Some general and simple thing to try to do to narrow down where your issue lies.

To rule out agent specific issues:
  • How many clients have you deployed caching to thus far?
  • Are all the locations and clients having this issue or is this a one off client/location?
  • Have we tried a different agent?
  • Have we tried an agent that has POSH 5.1 installed currently?
post back your findings.

Post Reply

Return to “Habitat”