--- type: Meeting Status: Archived _archived: true --- Meeting notes Incident management - Mariusz reported receiving multiple daily Splunk alerts about disk usage, resulting in duplicate incidents that affect statistics. - Mariusz fixed eight database-related disk usage cases and escalated two cases to the Windows team for disk extension. - Mariusz explained that the Windows team requires a five-day advance change request for disk extensions, causing delays in resolving incidents. - Mariusz clarified that duplicate daily incidents are being generated from Zabbix via Splunk integration, not from Splunk observability alerts. - Oleksandr explained that Splunk updates existing incidents instead of creating new tickets, while Zabbix integration creates multiple tickets due to its configuration. - Mariusz described ongoing issues with duplicate incident tickets generated daily, which negatively affect team statistics. System integration - Mats mentioned ongoing testing of ServiceNow integration, which may be related to the incident creation process. - Mariusz explained that some servers in the failover database cluster were never attached to Splunk due to port conflicts, despite the Splunk agent being installed. - Mariusz described ongoing errors when attempting to update monitoring agents or add servers to Ansible, including connection failures and outdated agents. - Kamil offered to mark servers missing in Splunk in Mariusz's file to help prioritize which servers to skip for now System monitoring gaps - Mariusz highlighted that several servers are not monitored by basic infrastructure templates, making it impossible to add database monitoring. - Mats reported sending a ticket to Lashek after discovering many Windows servers missing from monitoring, while Linux servers were fully accounted for. - Mats reported sending a ticket to Lashek after discovering many Windows servers missing from monitoring, while Linux servers were fully accounted for. System troubleshooting - Oleksandr suggested checking firewall settings and block lists as a possible cause for recent connection issues, but Mariusz noted the servers had worked a week ago. - Oleksandr explained that connection issues may be due to disabled services or local firewall settings on Windows machines, and suggested testing with the Windows team Follow-up tasks | Task | Assigned to | Due date | Bucket | | :-------------------------------------------------------------------------------------------------------------------------------------- | :---------- | :------- | :----- | | Mariusz stated they will discuss the issue of excessive incident tickets with Adrian to address the impact on team statistics (Mariusz) | | | | | Contact the Windows team to resolve Ansible connection issues for specific servers (Mariusz) | | | |