FAQ
| This is a
LIVE
service |
Changelog
Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
D
DevOps Division Guidebook
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Requirements
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Admin message
GitLab has been upgraded. See what's new in the
Changelog
.
Show more breadcrumbs
Information Services
DevOps
General Documentation
DevOps Division Guidebook
Commits
403e2d6d
Commit
403e2d6d
authored
3 years ago
by
Robin Goodall
Browse files
Options
Downloads
Patches
Plain Diff
Update dev group deployment
parent
c2ec3e37
No related branches found
Branches containing commit
No related tags found
Tags containing commit
1 merge request
!100
Update dev group deployment
Pipeline
#126656
passed
3 years ago
Stage: test
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
docs/notes/ucs-dev-group-services.md
+80
-39
80 additions, 39 deletions
docs/notes/ucs-dev-group-services.md
with
80 additions
and
39 deletions
docs/notes/ucs-dev-group-services.md
+
80
−
39
View file @
403e2d6d
...
...
@@ -18,49 +18,90 @@ The services run by DevOps currently includes:
The majority of the UCS dev group services are clustered using
[
Pacemaker
](
https://clusterlabs.org/pacemaker/man/
)
.
To make a release without incurring downtime the following steps can be taken:
To make a release without incurring downtime the following steps can be taken.
Examples, refer to Lookup service (
`ucs-ibis`
package) and need substituting as
appropriate.
### Start with the standby node
!!! note
Which node is on standby can be determined by running
`crm_mon -1`
or looking at the service's
`/adm/status`
page.
1.
Put the cluster into
[
maintenance mode
](
https://xahteiwi.eu/resources/hints-and-kinks/maintenance-active-pacemaker-clusters/
)
,
this ensures pacemaker does not try to make changes to the cluster in response to a node being
unavailable.
```
crm configure property maintenance-mode=true
```
2.
On the standby node (can be determined by running
`crm_mon`
) run the software upgrade, in the
case of Lookup this would look like:
```
zypper ref
zypper lu
zypper up ucs-ibis
service tomcat6 stop
service tomcat6 start
```
3.
Once the upgrade has been successfully deployed to the standby node, move the service
over to the current standby node by editing the pacemaker configuration:
```
crm configure edit
# shift node weights so that the current standby node is the preferred service
# verify that the service has moved to the previous standby:
crm_mon
```
4.
Make the software upgrade on the node which has now become the standby.
5.
Reconfigure the cluster to its previous state, and move out of maintenance mode:
```
crm configure edit
# shift node weights so that the current standby node is the preferred service
# verify that the service has moved to the previous standby:
crm_mon
# move out of maintenance mode
crm configure property maintenance-mode=false
```
```
crm configure property maintenance-mode=true
# Check for "unmanaged" status
crm_mon -1
```
2.
Release the lock on the package.
```
# List locked packages
zypper ll
# Release lock on appropriate package
zypper rl ucs-ibis
```
3.
Run the software upgrade.
```
# Refresh repositories
zypper ref
# List updates available
zypper lu
# Update application specific package
zypper up ucs-ibis
# Restart tomcat (a single "restart" doesn't always work)
service tomcat6 stop
service tomcat6 start
```
4.
Check the service is running on the node with update.
-
see
`https://{node url}/adm/status`
-
check application functionality
5.
Move the service out of maintenance mode.
```
crm configure property maintenance-mode=false
# Check for removal of "unmanaged" status
crm_mon -1
```
6.
Reapply lock to package.
```
# Release lock on appropriate package
zypper al ucs-ibis
# List locked packages to check
zypper ll
```
### Move to current live node
1.
Move service to already updated standby node
```
crm configure edit
# shift node weights so that the current standby node is the preferred service
# verify that the service has moved to the previous standby:
crm_mon -1
```
2.
Repeat steps 1 to 6 above. i.e.
-
put the service back in maintenance mode
-
unlock the package
-
update the package
-
check success
-
remove the service from maintenance mode
-
relock the package
3.
Move service to back to this node
```
crm configure edit
# shift node weights so that this nde is the preferred service
# verify that the service has moved back:
crm_mon -1
```
Following the above should allow a software upgrade to be deployed without any downtime. These
steps will not work if the upgrade includes a breaking change to an external data source, e.g.
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment