3rd Party Application Management

[dbitters]

Does rabbit mode on weekly still try a few times a day or only on the day I picked? I wasn't sure if rabbit scanned multiple times a day on both daily and weekly or not.

I know the weekly turtle doesn't work, but I'll double-check weekly rabbit.

I'm also wondering if there is an adjustable time that this all kicks off. All the scans and installs seem to kick off at 6:30 PM. I wasn't sure if that's by design or not.

thanks for the help.

[Cubert]

Here is how the turtle and rabbit schedules work. Each case represents the Hour of the Day (Case 2 = 2:30AM, Case 14 = 2:30 PM), we run on the half hour. You will see that we test for different states of an agent to determine if scheduling applies at the times given. So although a service may run 5 times a day, it may find zero agents to match its function and will do nothing at all. Install framework is the largest of these functions. It looks regularly for new agents to appear and will attempt to start prepping agents as they become available and enabled.

Rabbit:

Code: Select all

         
                     Case 1
                        DeployFramework()
                    Case 2
                        CacheApps()
                        DeployFramework()
                    Case 4
                        RunAPPManageMaintenance()
                        MasterVersioning()
                        DeployFramework()
                    Case 8
                        DeployFramework()
                    Case 10
                        DeployFramework()
                        VersionChecks()
                    Case 12
                        DeployFramework()
                    Case 14
                        DeployFramework()
                        AgentSYNC()
                    Case 16
                        DeployFramework()
                    Case 19
                        DeployFramework()
                    Case 21
                        VersionChecks()
                        DeployFramework()

Turtle:

Code: Select all

                    
                    Case 1
                        DeployFramework()
                    Case 2
                        CacheApps()
                        DeployFramework()
                    Case 4
                        RunAPPManageMaintenance()
                        DeployFramework()
                        MasterVersioning()
                    Case 9
                        VersionChecks()
                        DeployFramework()
                    Case 15
                        DeployFramework()
                        AgentSYNC()
                    Case 20
                        VersionChecks()
                        DeployFramework()

DeployFramework() - Looks for online agents that show in database that choco.exe is not yet installed. Schedules script to install framework.

CacheApps() - Schedules the App caching agents whom are online to test the cache and update where needed.

RunAPPManageMaintenance() - This is an internal test and cleanup of the automate database tables for App Manager tool. Removes old and stale data if any exists.

MasterVersioning() - This is a script scheduled to the caching agents to update the master App list versions with current listed version numbers from the Choco Community Repository.

AgentSYNC() - This is the meat and potatoes, this is a scheduled script on all enabled and online agents to sync applications against the cache. This is where agents will be called to update applications.

VersionChecks() - This schedules script on agent to collect the current versions of applications as the local framework see it currently. Different from MasterVersion as this is all the agents testing themselves for versions where Master is testing Choco Community Repo for current versions available.

This can happen daily, weekly or monthly by selecting the correct Schedule from the Client View.

Hope this helps clarify how, what and when Automation happens in Habitat Application Manager.

[Cubert]

I needed to mention the following,

Several of the services above run daily based on mode no matter what schedule you have set in client area. Run Maintenance, MasterVersion, AppCache and InstallFramework run daily whether or not a schedule is set to weekly or monthly.

It is AgentSync that is mainly controlled by scheduler. The other agent facing service is versioning and that is just a quick collection of local stats on agent and is of little weight on agents.

[dbitters]

After doing some more testing, I can verify that the daily schedule is working fine, but the Weekly schedule is having issues with turtle mode.
I was planning on using turtle mode on a weekly schedule to cut down on unneeded scripts running to avoid excessive daily scripts and logging. If rabbit mode on a weekly schedule only kicks on once a week, then that's probably fine, and I'll stick to that. I just didn't want rabbit mode from running additional scripts all week long.

Your recommendation for running a truncate on the app table is a good idea as I was noticing legacy information that didn't get cleaned up automatically.

Habitat app table truncate.PNG (12.74 KiB) Viewed 2101 times

I added this to the Habitat Database Maintenance tool since I have a hosted server and running command directly is usually an issue. Maybe there could be a consideration to add some built-in maintenance functions in that console to automatically truncate certain areas after data becomes old when manually toggled. Might be worthwhile to make it easier to clean things out every 6 months to drop any stale data. However, the disable/enable button on individual functions does not work, so you have to enable and disable entire categories to disable a function.

I'm also noticing that two customers are refusing to use the caching agent on its configured sites. On my initial tests using the test script on those two customers, the test actually runs successfully. Due to that problem, each system tries to go outbound directly and get a "too many requests" as was the issue before the caching agent. I'm wondering what the best way is to reset that since something is odd with that since it passed multiple test script checks on a few systems in those sites, but the automatic weekly scripts don't seem to try to use the caching agent, even though the icon shows up beside the computer name. The other customers are fine, so I think it's just something stuck from previously using the old version.

Thanks again for your help.

[Cubert]

I will have a look at the turtle on weelky issue to see what's up and if I need to correct something.

As for the other issue, sounds like something odd with that client. Maybe disable and reenable the client will reset it.

[dbitters]

I think the development of Habitat and the App manager is doing great. However, I do have several items that I thought could use some attention to see if consideration could be taken on them.

Item 1.
There is still something wrong with the schedule. The schedule doesn't appear to be activating when switching back and forth between daily and weekly mode, even when it says it has saved. When I look at the LT script logs, I can see that the Habitat App manager scripts don't actually kick off as corresponding to the schedule type. The problem can be in either direction. I think that is part of the problem I was previously reporting. Honestly, I think it would be better to have three radio boxes where you could select Daily, Weekly, or Monthly and then press save. Having a radio button for all three would be more intuitive and probably more reliable in the long run as there have been problems with the current method not working as expected. For example, I switched two customers to daily at the beginning of the week to test this, and the schedule clearly says daily, but the last script run was a week ago on the previous weekend scheduled day when the schedule was weekly mode on the weekend. But when switching back to daily, the scripts aren't activating on daily. Previously I was seeing this in reverse when the scripts would not kick off on the day of the week the weekly schedule was set to. I was also still having issues with turtle mode on either, but if I just stick to Rabbit mode, that would be fine as long as the glitch in the schedule is fixed, but if I switch back and forth a few times and restart the database, then things kick back into gear, or so it seems.

Item 2.
The caching agent system can't seen to scan itself, list versions or update itself. The caching agent actually has blank data for all the columns as it is not populating any info by default. To correct that, I can simply right-click on the caching agent and select run version scan, then update now. That's not a huge deal, but I have confirmed that issue on several customers, so that may want to get looked at so it is also automatic. I think this issue is related to a problem when it tries to clear the cache upon completion for just the Caching Agent system.

Item 3.
There are issues downloading some package files. Part of this may be just an issue with bad packages, but I wanted to run this by you just in case. This issue deals with specifying variables in the download source link that is not valid. For example, the issue with Firefox is that it includes the "locale" variable at the end of the download line, which is not valid.
Download Failed: https://download.mozilla.org/?product=f ... g=${locale}
If I remove the "&lang=${locale}" at the end of the line, it works. It seems like it might be a Firefox package or site issue, but if that is related to the parsing from Habitat in any way I wanted to bring it up.
Here is a list of various package download issues I am seeing, which may be issues with those packages or possibly with Habitat.

Habitat cache.PNG (114.15 KiB) Viewed 1978 times

Item 4.
There are issues with the package download on the Caching Agent, even though the downloads almost always complete. This may be related to the issue of applications and versions not populating. Specifically, I am seeing "Access Denied to the path 'C:\Windows\LTSvc\Habitat\packages". This is shown on every application listed but only shows on the Caching Agent system.
I don't know why it would be downloading there because that is not the cache location, unless it is trying to copy from the "Script and template variables" Drive.
Here is an example.

Habitat cache access denied.PNG (38.08 KiB) Viewed 1978 times

To be more specific, on the caching Agent/Server every application triggers a "Checking repo cache for [application_name]"
On every package, it always triggers the following error every time it goes through a repo cycle.
"Cacher status -> [Directory 'C:\Windows\LTSvc\Habitat\packages' does not exist.]"
That is not the repo cache directory, so it doesn't make sense why it's checking the default LT Habitat location for a packages directory as that is not the configured Habitat packages directory. The configured packages directory as specified at the LT site level is where the packages directory exists.

Item 5.
Would it be possible to include a method to reset all application data and versions at a global level and at a customer level? I say this because sometimes there might be situations where there are major application changes across either a particular customer or all customers. It would be more reliable to start over with a new scan of new application data and versions and to truncate all old data. The second reason would be after any technical issues such as a modification to the Habitat or the application list or other issues that require troubleshooting and require a reset. Being able to do this at a customer level would be very convenient, especially when a particular customer just doesn't work. I'm also thinking at the global level for the same reason if there are major issues, as this would be a lot more convenient than adding a truncate table command from the SQL maintenance tool.


Item 6.
Would it be possible for a repository rescan button on the Application management main console instead of waiting overnight for that particular script of Habitat to kick off. I say this because if I'm troubleshooting something and I change the list of global applications, the repository field will say none until 5AM. This can be problematic if I'm troubleshooting something. I'd like to be able to press a button that allows the global application list to rescan the Chocolatey repository and record the version information. I'd also like to see a reset button that clears all the values to none so I can then press a rescan button to recheck all the versions, which can wipe out any legacy applications that no longer have valid repository information in Chocolatey. This would be helpful when troubleshooting applications and removing stale applications that are legacy or if there is any problem with the list. For example, this screenshot shows applications I was troubleshooting, but I have to wait overnight to see if the version populates, confirming it is working.

Habitat Search.PNG (47.18 KiB) Viewed 1960 times

Item 7.
Is there a way for Habitat to truncate applications that no longer exist when it does its rescan at the customer level? I ask because we removed an application a few weeks ago from all systems of a customer, but yet Habitat still says that the application is still installed on each system, with the version at the time, but in fact, that software is not even installed. I think this is critical to ensure that people have faith in the data being reported to ensure it can be trusted. If that function isn't possible, then that customer level reset button is what I would need so I can issue a reset to correct such anomalies to ensure accurate data.

Item 8.
Is there a way to include the application's actual name on the Managed Applications list at the customer level, which is resolved and shown on the main console at the global level during the search? At the customer level, it currently shows the application package name, the last update, installed version, and repo version. But the package name is being listed as the application name. However, it would be great if the actual application name could be a column such as the first column and then the column for the package name. This is very helpful when the package names don't line up with the application name, which is the case with some package names. This is also helpful when the same package name is used for multiple installed applications or when troubleshooting variances in application names where it might actually require a different package, and you can't see what application is being targeted.

Item 9
In some situations, the installed version column is still not showing up, and there is an error stating, "Application source is unavailable". These systems pass all the script checks, and some systems in the same site are showing up correctly. See the screenshot below to so you can see what I mean.

Habitat app error.PNG (43.43 KiB) Viewed 1929 times

Item 10.
Is there a way for Habitat to update the “Programs and Features” / “Add Remove Programs”, so it properly reflects the updated version after the applications have been updated? I noticed that even though the applications are being updated, Windows still sees the applications as having the old versions. Also, when Labtech does its inventories, it also still shows the applications having the old versions. This is problematic when doing audits for auditors because we are not in compliance unless the inventory lists show the application versions as current. The auditors then verify this information on systems by spot-checking the “programs and features” version list or issuing a remote query to query the system to pull the software installation status and versions that also show the old version. I am hoping that Habitat can find a way to update Windows with the updated application version when it completes the application upgrade. Since this is a major issue with auditors, I am hoping there is some way we can deal with this setback.

Thanks for all the help through these remaining issues.

[Cubert]

Wow, that must of took a lot of thought and effort to define. Thank you!

I have read through the 10 items briefly and I didn't see anything that's not doable in some fashion.

Let me take this and break it down in to separate tasks and issues and lets see what we can do.

[Cubert]

New in Application manager in Habitat Build 1.0.1.14 is due out by weeks end. We started working on your list and here is what we have thus far.

In this build we have new updates for Application Manager.

#1 There are now three radio boxes where you could select Daily, Weekly, or Monthly when scheduling updates.

Capture.PNG (17.93 KiB) Viewed 1869 times

#2 New method to reset all application data Globally and Per Client.
In both the global and client views, if you mouse click the Habitat logo while holding down the CTRL key you will receive a pop up message. Select yes to clear all data from database for all clients or the client you currently managing.

Capture1.PNG (36.26 KiB) Viewed 1869 times

#3 New repository rescan versions, deploy framework and App update controls on the Application management main console. These functions control all clients currently enabled.

Capture11.PNG (32.31 KiB) Viewed 1869 times

#4 Select and truncate applications that no longer exist in Client console.
Actually we place a new menu item in the applications view to allow you to uninstall the application. It will push chocolatey to uninstall and will remove data from data base. Safe to use even if you already removed application from agent.


#5 Now Include the application's actual name on the Managed Applications list.
We now include the actual application name that the packages is updating in the app list. If your agent is updating JRE8 then you will see the Java 8 Runtime ver 8.x.x.xas it would appear in Windows Software list.

[dbitters]

This all looks fantastic. I am eager to test it out. Thanks for all the consideration and the work.

[dbitters]

I actually have an additional item. But as a side topic, for whatever reason when I'm writing a long post, my drafts to this forum can get deleted if I don't click post before the timer expires and logs me out. So I ended up writing the last long post three times before switching to Word. I missed two items and just now remembered one more.

Item 11.
The Database Maintenance component of Habitat is great, but there is a minor bug. If I add a function to a group, I can not disable that function without disabling the group. The option is there to enable/disable, but it's actually only for enabling, as disabling does not do anything. Therefore to add functions that you want to keep for later, you need to make one function per group and then disable the groups since disabling the functions has no effect. I can enable a function, but once it's enabled, it can't be disabled. It may be better to make one enable option and one disable option.

Thanks.