Hey Folks,

Hub 3 might be a bit iffy today. I'm trying to track down 2 issues that I cant seem to create in my test envrionment.

I've put it in debug mode, and may need to keep increasing debugging in particular areas until I find it.

Hopefully regular programming will return soon.


...ëîåï
--- SBBSecho 3.20-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)

Re: Hub 3
By: deon to All on Thu Jun 27 2024 08:11 am

Howdy,

Hub 3 might be a bit iffy today. I'm trying to track down 2 issues that I cant seem to create in my test envrionment.

So the first issue I tracked down and fixed (clrghouz had mail for an upstream node, but for some reason wasnt sending it).

The 2nd issue has me stumped. I've been doing some updates after the last couple of days, and I started seeing session failures.

I've spent the last 24-36 hrs debugging and tracing and cant figure it out, so I'm now wondering if it is the upstream system (or my testing hasnt triggered the sequence for the problem).

I can see mail flowing between *most* nodes connected to Hub 3, with the exception of 3:
* 21:1/100 - although a test message just got a response from 6/100 which is via 1/100.
* 21:3/110
* 21:3/119
* 21:3/170

The only commonn thing I can see is that those nodes are using binkd (might be a coincidence?)

The issue I'm seeing is when those poll in, I'm sending a mail packet, and while waiting for the M_GOT, I'm getting random 3 chars (and those characters make me wait for more data which never arrives. After 5 mins, the binkd session times out.

I think if I poll outbound, I can successful send mail (not 100% confident that this is right).

Anybody else having issues?

Jay/Stephan/Drew if you get this I'd be keen to see your logs and maybe turn it up (especially if you are successful getting mail from other systems).


...ëîåï
--- SBBSecho 3.20-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)

On 27 Jun 2024 at 05:33p, deon pondered and said...

The 2nd issue has me stumped. I've been doing some updates after the last couple of days, and I started seeing session failures.
I've spent the last 24-36 hrs debugging and tracing and cant figure it out, so I'm now wondering if it is the upstream system (or my testing hasnt triggered the sequence for the problem).

Hi there

I've sent you some logging from my end at 1/100 which I hope helps :)

Kerr Avon [Blake's 7] 'I'm not expendable, I'm not stupid and I'm not going' avon[at]bbs.nz | bbs.nz | fsxnet.nz

--- Mystic BBS v1.12 A48 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 28 Jun 2024, deon said the following...

Jay/Stephan/Drew if you get this I'd be keen to see your logs and maybe turn it up (especially if you are successful getting mail from other systems).

Sure, I can send some logs your way.

I'm currently set to loglevel 4 but can ramp that up for a bit as well.


Jay

... A baker's calendar has 13 months

--- Mystic BBS v1.12 A49 2024/05/29 (Linux/64)
* Origin: Northern Realms (21:3/110)

Re: Re: Hub 3
By: Warpslide to deon on Thu Jun 27 2024 08:12 am

Howdy,

Jay/Stephan/Drew if you get this I'd be keen to see your logs and maybe turn it up (especially if you are successful getting mail from other systems).

Sure, I can send some logs your way.

The verbose logging would be helpful, thanks.

In the mean time, I've spun up an old instance of clrghouz and are redirecting you and the others affected to it - so mail appears to be flowing for you now. Those not affected are on the latest clrghouz.

Just need to figure how I broke it. So I'll be progressing each commit until I find the commit that broke it - please bear with me if you see mail errors while I'm chasing this. (Least I know which commit works.)

(The wonders of containers and git that make this kinda easy to debug.)


...ëîåï
--- SBBSecho 3.20-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)

On 28 Jun 2024, deon said the following...

I can see mail flowing between *most* nodes connected to Hub 3, with the exception of 3:
* 21:1/100 - although a test message just got a response from 6/100
which is via 1/100.
* 21:3/110

I don't know if it's related/relevant but when I poll you I'm using IPv6 & SSL in binkd:

node 21:3/100@fsxnet -md -6 -pipe "openssl s_client -6 -quiet -alpn binkp -connect *H:*I" n3.z21.bbs.dege.au:24553 <password>


Jay

... Some things have got to be believed to be seen

--- Mystic BBS v1.12 A49 2024/05/29 (Linux/64)
* Origin: Northern Realms (21:3/110)

Re: Re: Hub 3
By: Warpslide to deon on Thu Jun 27 2024 08:47 am

Howdy,

I don't know if it's related/relevant but when I poll you I'm using IPv6 & SSL in binkd:

Not related unfortunately. I have IPv6 and IPv4 nodes failing, as well as TLS and non-TLS.

The only common thing is binkd strangely.


...ëîåï
--- SBBSecho 3.20-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)

Re: Re: Hub 3
By: deon to Warpslide on Thu Jun 27 2024 10:40 pm

Howdy,

Just need to figure how I broke it. So I'll be progressing each commit until I find the commit that broke it - please bear with me if you see mail errors while I'm chasing this. (Least I know which commit works.)

OK, not sure how I broke it, reverted the patch that did it and we are back in action. Strange it was only affecting inbound connections (when that patch was addressing outbound ones), and only binkd systems.

Anyway, back to regular programming - let me know if you notice anything odd.


...ëîåï
--- SBBSecho 3.20-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)