Issue
Today I was working on a lab environment and encountered the following issue:
- Workflows did not show any System Log output. Events were working fine.
The environment consisted of a vRA 7.4 appliance with an embedded Orchestrator.
To test it out I created a workflow which did some crazy logging:
- No logs were showing, not with scriptable tasks neither with the built in System Log components.
- Events were still working fine.
Below some screenshots:
Analysis
Restarting the Orchestrator Client
My first attempt was to restart the Orchestrator Client, closing & opening it from scratch. However this did not resolve the issue. #TooEasy
Checking the logs!
I decided to SSH into the vRA appliance and check the logs by executing the following command:
tail -f /var/log/vmware/vco/app-server/catalina.out
The logs were a bit more conclusive:
There appeared to be an issue with obtaining a lock / the locking mechanism for the lucene environment.
Reading into it on the internet, I tried the following solution:
Solution
- Stop vco-server by executing the following command:
service vco-server stop
- Remove all occurences of scripting.log_lucene:
rm -rf /var/log/vco/app-server+scripting.log_lucene*
- start vco-server
service vco-server start
- Test the workflows!
Now everything seemed to be working fine! And /var/log/vmware/vco/app-server/catalina.out showed a beautiful sight!
Sources: