Re:
By: to on Thu Mar 28 2019 01:02 am

So, I read your reply to my message yesterday. It came up fine, with header information and everything. I didn't have time to pull the data, and thought to myself "I'll get back to it later".

Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

If you disable DDMessageReader, do you still have an issue?

I'm going to have to give that a shot. Thus far, I think the pattern seems to be that the header disappears when the message is marked as read by the recipient. I'm not 100% on that, though, I'll have to do some more digging.

Well it's certainly possible to import messages without message-IDs. Perhaps if you ran "chksmb -e" you'd see a pattern with the messages that are missing message-IDs and identify the cause. In any case, it's not a fatal issue normally.

here's my current chksmb -e output on dove-general and dove-syncops, the two echos that I've seen the issue crop up.

https://pastebin.com/YacRr6XV

After that, I tried a base fixsmb on dove-syncops, and here was the output:

[root@bbs subs]# /sbbs/exec/fixsmb dove-syncops

FIXSMB v2.10-Linux (rev 1.46) SMBLIB 2.60 - Rebuild Synchronet Message Base

Opening dove-syncops
99% #9698 (703F20)
(704020) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7356448
99%
(704120) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7356704
99% #9708 (705820)
(705920) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7362848
99%
(705A20) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7363104
100%
Sorting index...
Re-writing index...

Saving message base status (9716 total messages).
Closing message base.
Done.

DaiTengu

... Only dull people are brilliant at breakfast.

---
þ Synchronet þ War Ensemble BBS - The sport is war, total war - warensemble.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: DaiTengu to Digital Man on Fri Mar 29 2019 10:59 am

Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

I haven't seen that problem on my BBS with my reader, and haven't heard about this from other sysops using my reader either. The way it marks a message as read is by loading the message header without expanded fields (using MsgBase.get_msg_header()), applying the MSG_READ attribute, and then saving the header back to the messagebase (using MsgBase.put_msg_header()). I wouldn't expect that to corrupt the header.

Nightfox

---
þ Synchronet þ Digital Distortion: digitaldistortionbbs.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

I haven't seen that problem on my BBS with my reader, and haven't heard about this from other sysops using my reader either. The way it marks a message as read is by loading the message header without expanded fields (using MsgBase.get_msg_header()), applying the MSG_READ attribute, and then saving the header back to the messagebase (using MsgBase.put_msg_header()). I wouldn't expect that to corrupt the header.

I wouldn't either, but right now it's the only hypothesis I have that fits my issues (header displays fine, once it's read, header disappears).

I'm replying to this message just fine, the header was there, but it's the first time I viewed your message.

EDIT: Yep, header disappeared when I went back and veiwed the messages in this area again.




DaiTengu

... Ahh! Come on Erick, just this one last little feature!


Mike

---
þ Synchronet þ War Ensemble BBS - The sport is war, total war - warensemble.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

Re: Corrupt Message Headers
By: DaiTengu to Digital Man on Fri Mar 29 2019 10:59 am

Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

I haven't seen that problem on my BBS with my reader, and haven't heard about this from other sysops using my reader either. The way it marks a message as read is by loading the message header without expanded fields (using MsgBase.get_msg_header()), applying the MSG_READ attribute, and then saving the header back to the messagebase (using MsgBase.put_msg_header()). I wouldn't expect that to corrupt the header.

It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

digital man

This Is Spinal Tap quote #22:
David St. Hubbins: Here lies David St. Hubbins... and why not?
Norco, CA WX: 65.8øF, 35.0% humidity, 0 mph SSE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: DaiTengu to Digital Man on Fri Mar 29 2019 10:59 am

Re:
By: to on Thu Mar 28 2019 01:02 am

So, I read your reply to my message yesterday. It came up fine, with header information and everything. I didn't have time to pull the data, and thought to myself "I'll get back to it later".

Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

If you disable DDMessageReader, do you still have an issue?

I'm going to have to give that a shot. Thus far, I think the pattern seems to be that the header disappears when the message is marked as read by the recipient. I'm not 100% on that, though, I'll have to do some more digging.

Well it's certainly possible to import messages without message-IDs. Perhaps if you ran "chksmb -e" you'd see a pattern with the messages that are missing message-IDs and identify the cause. In any case, it's not a fatal issue normally.

here's my current chksmb -e output on dove-general and dove-syncops, the two echos that I've seen the issue crop up.

https://pastebin.com/YacRr6XV

After that, I tried a base fixsmb on dove-syncops, and here was the output:

[root@bbs subs]# /sbbs/exec/fixsmb dove-syncops

FIXSMB v2.10-Linux (rev 1.46) SMBLIB 2.60 - Rebuild Synchronet Message Base

Opening dove-syncops
99% #9698 (703F20)
(704020) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7356448
99%
(704120) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7356704
99% #9708 (705820)
(705920) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7362848
99%
(705A20) smb_getmsghdr returned -103:
smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7363104
100%
Sorting index...
Re-writing index...

Saving message base status (9716 total messages).
Closing message base.
Done.

And after that, does chksmb still report errors? If so, I recommend using smbutil to repair the msgbase:
http://wiki.synchro.net/util:smbutil#examples

digital man

This Is Spinal Tap quote #37:
David St. Hubbins: We are Spinal Tap from the UK - you must be the USA!
Norco, CA WX: 65.0øF, 40.0% humidity, 2 mph ESE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

Yep.

Synchronet BBS for Linux Version 3.17 Debug

Revision C Mar 25 2019 08:44 SMBLIB 2.60 GCC 4.8.5

Copyright 2019 Rob Swindell - http://www.synchro.net

JavaScript-C 1.8.5 2011-03-31

GLIBC 2.17 cryptlib 3.4.4 (3440)

Linux 4.18.16-x86_64-linode118 x86_64

The problem started a week or so ago, I think? I noticed it's happening in the fidonet echos as well, as one of my users recently started posting there, and those messages are missing header information as well.

Right now I'm reading/replying through the web interface as a lazy way to avoid disabling DDMsgReader globally.

---
þ Synchronet þ War Ensemble BBS - The sport is war, total war - warensemble.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Digital Man to Nightfox on Sat Mar 30 2019 09:59 pm

It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

Viewing through the web interface seems to mark messages as "read", but the minute I went back and re-read them with DDMsgReader, the header information went away.

I've now disabled DDMsgReader completely on my board, When the next message comes in addressed to me, I'll see if it happens with the stock Synchronet reader.

DaiTengu

... The word 'meaningful' when used today is nearly always meaningless.

---
þ Synchronet þ War Ensemble BBS - The sport is war, total war - warensemble.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: DaiTengu to Digital Man on Sun Mar 31 2019 11:33 am

Re: Corrupt Message Headers
By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

Yep.

Cool. I was asking Nightfox however, since he can't seem to reproduce the issue. Perhaps it was a recently-introduced bug.

digital man

This Is Spinal Tap quote #46:
"Not an Exit" - we don't want an exit. Well that's true.
Norco, CA WX: 79.0øF, 19.0% humidity, 0 mph SSW wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Digital Man to Nightfox on Sat Mar 30 2019 09:59 pm

I haven't seen that problem on my BBS with my reader, and haven't
heard about this from other sysops using my reader either. The way it
marks a message as read is by loading the message header without
expanded fields (using MsgBase.get_msg_header()), applying the
MSG_READ attribute, and then saving the header back to the messagebase
(using MsgBase.put_msg_header()). I wouldn't expect that to corrupt
the header.

It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

I wasn't, but just now I updated to the latest binaries (built today, 3/31). Now I'm seeing an error with my message reader - After reading a new message to me, my reader will add the MSG_READ attribute and save it back to the msgbase. With the latest binaries, I'm now seeing this error when calling MsgBase.put_msg_header():

!JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529: Error: Error -110 adding SENDERNETADDR field to message header

I wasn't seeing that error before. I'm not sure what would cause that error. My reader caches the message headers (it has an array of saved message headers) and it was applying the attribute to the cached header and then trying to save it to the msgbase. As a workaround, I tried the following:
- Load the message header from the messagebase (without expanded fields)
- Apply the new attribute (MSG_READ)
- Save that header back to the msgbase
However, on the first tep, when reading the header from the msgbase, it got a null object.

Nightfox

---
þ Synchronet þ Digital Distortion: digitaldistortionbbs.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Nightfox to Digital Man on Sun Mar 31 2019 05:21 pm

I haven't seen that problem on my BBS with my reader, and haven't
heard about this from other sysops using my reader either. The way
it marks a message as read is by loading the message header without
expanded fields (using MsgBase.get_msg_header()), applying the
MSG_READ attribute, and then saving the header back to the
messagebase (using MsgBase.put_msg_header()). I wouldn't expect that
to corrupt the header.

It's always possible I hosed something up in a recent commit to CVS.
Are you running the latest and greatest?

I wasn't, but just now I updated to the latest binaries (built today, 3/31). Now I'm seeing an error with my message reader - After reading a new message to me, my reader will add the MSG_READ attribute and save it back to the msgbase. With the latest binaries, I'm now seeing this error when calling MsgBase.put_msg_header():

!JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529: Error: Error -110 adding SENDERNETADDR field to message header

I wasn't seeing that error before. I'm not sure what would cause that

Now that I've updated to today's binaries, I think I'm seeing the same issue as DaiTengu now. I read one of Vk3jed's messages to me, and after my reader
tried to mark his message as read, now the header of the message from Vk3jed seems to be messed up. In my reader's reader interface, now it says it's written to 'All' instead of me, and the 'From' and 'Subject' are empty. In my reader's message list (where it displays the 'from', 'to', and subject, those are empty after my reader tried to apply the 'read' attribute to the header in the msgbase.

Nightfox

---
þ Synchronet þ Digital Distortion: digitaldistortionbbs.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Nightfox to Digital Man on Sun Mar 31 2019 05:36 pm

!JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529:
Error: Error -110 adding SENDERNETADDR field to message header

I saw that same error in my error log a couple days ago, but I haven't seen it since.

Mar 29 16:28:37 bbs sbbs: term Node 1 <DaiTengu> !JavaScript ../xtrn/DDMsgReader/DDMsgReader.js line 1525: Error: Error -110 adding SENDERNETADDR field to message header

DaiTengu

... Success is a public affair. Failure is a private funeral.

---
þ Synchronet þ War Ensemble BBS - The sport is war, total war - warensemble.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Corrupt Message Headers
By: Nightfox to Digital Man on Sun Mar 31 2019 05:36 pm

Re: Corrupt Message Headers
By: Nightfox to Digital Man on Sun Mar 31 2019 05:21 pm

I haven't seen that problem on my BBS with my reader, and haven't
heard about this from other sysops using my reader either. The way
it marks a message as read is by loading the message header without
expanded fields (using MsgBase.get_msg_header()), applying the
MSG_READ attribute, and then saving the header back to the
messagebase (using MsgBase.put_msg_header()). I wouldn't expect that
to corrupt the header.

It's always possible I hosed something up in a recent commit to CVS.
Are you running the latest and greatest?

I wasn't, but just now I updated to the latest binaries (built today, 3/31). Now I'm seeing an error with my message reader - After reading a new message to me, my reader will add the MSG_READ attribute and save it back to the msgbase. With the latest binaries, I'm now seeing this error when calling MsgBase.put_msg_header():

!JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529: Error: Error -110 adding SENDERNETADDR field to message header

I wasn't seeing that error before. I'm not sure what would cause that

Now that I've updated to today's binaries, I think I'm seeing the same issue as DaiTengu now. I read one of Vk3jed's messages to me, and after my reader tried to mark his message as read, now the header of the message from Vk3jed seems to be messed up. In my reader's reader interface, now it says it's written to 'All' instead of me, and the 'From' and 'Subject' are empty. In my reader's message list (where it displays the 'from', 'to', and subject, those are empty after my reader tried to apply the 'read' attribute to the header in the msgbase.

I reproduced and fixed the problem (in js_msgbase.c) - it was a bug introduced in the most recent change to the MsgBase.put_msg_header() implementation.


digital man

Synchronet "Real Fact" #74:
Vertrauen went online (as a WWIV BBS running on a 10MHz PC-XT clone) in 1988. Norco, CA WX: 80.0øF, 24.0% humidity, 10 mph NE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re:
By: to on Mon Apr 01 2019 04:15 pm

!JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529:
Error: Error -110 adding SENDERNETADDR field to message header

Now that I've updated to today's binaries, I think I'm seeing the

same issue as DaiTengu now. I read one of Vk3jed's messages to me,

I reproduced and fixed the problem (in js_msgbase.c) - it was a bug introduced in the most recent change to the MsgBase.put_msg_header() implementation.

I updated to today's binaries, and the issue seems to be fixed now.

Nightfox

---
þ Synchronet þ Digital Distortion: digitaldistortionbbs.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)