502 Bad Gateway

Hi! I’m still fighting my way through the Mattermost import. Yesterday evening I got to a point at which the import actually started, but triggered some errors. Right after that, reloading the Mattermost site (readingmatter.info) started giving me 502 Bad Gateway errors from nginx. I wound up removing the environment and starting again from the beginning – but was still getting the 502s. That had resolved this morning, so I started setup – and seem to have triggered a new round of 502s. Is there something about the ways I’m using the load balancer, or something up with nginx that I should know about?

1 Like

Kathleen,

Let’s try you adding me to your instance again with jim@reclaimhosting.com so I can take a look. Hopefully that will work this time, but if I’ll report back here either way.

Invitation should be on its way!

I got the invitation and I tried removing the load balancer and running it from just the Docker container to no avail. I am seeing no difference between your environment and mine, and I can reproduce the Nginx errors. Any chance you captured the errors after import?

Not sure if @cblankenship has any ideas, so copying him here, but if now that I have access to your environment I can rebuild it from scratch and give it a shot, just send me the import file via some other means and I can report back in the morning given I’ll some time first thing tomorrow.

Hi Jim. Alas, I did not capture the errors, which caused the import to fail with no imported records. Still, it was a success because I got the import to start, which was further than I’d gotten before. The error had to do with a missing default channel in Mattermost – I’d deleted one of the pre-installed channels which turns out to have been a mistake. So my guess last night was that I’d triggered some kind of general database/container freakout – but I’m not sure why it’s ongoing, since I then entirely removed and rebuilt that environment…

ETA: I’m also fine with continuing to tinker with all this, though I’m sure y’all are ready for me to stop bombarding you with dumb questions. :slight_smile:

I took a glance at the logs and configuration of your docker containers, and it looks like the app itself is set to run on port 80 (HTTP), but it appears to be running on port 443 (HTTPS). This disconnect may be the cause of the the 502 error.

Do we change that in the Firewall or the nginx configuration? Or is that app specific?

As you can see, we are tinkering right alongside you, we’ll figure it out. I might have to try an import myself tomorrow to reproduce the 502 error, but @cblankenship’s point about running on http vs https might be worth looking at in your settings of Mattermost, but still trying to nail down where exactly those would be.

It looks like this would be with the app itself; I checked the firewall and it has all the appropriate rules and read that a container will be marked “unhealthy” if it’s not giving a response back to its monitor (which would be checking on the wrong port here). I’ll have to set up my own environment to see if I can spot where the setting to change the container from running on port 80 to port 443.

Within the application, if I could get to it, the Console has a page for web server settings, which includes a setting to forward port 80 to port 443. I set that to on. Is the problem that setting, or is it that the setting isn’t working? (I’d turn it off but I can’t get to the application given the nginx error. According the the Mattermost docs, most settings should be contained in config.json in the mattermost/config directory. I’m going to see if I can find the specific setting…

I found it, and that did it. nginx is now leaving me alone. Onward :slight_smile:

2 Likes

This is awesome news, and damn you are a trooper! Also, thanks for the bail out @cblankenship -you rule!