• Corrupt Message Headers

    From DaiTengu@1:103/705 to Digital Man on Friday, March 29, 2019 10:59:11
    Re:
    By: to on Thu Mar 28 2019 01:02 am

    So, I read your reply to my message yesterday. It came up fine, with header information and everything. I didn't have time to pull the data, and thought to myself "I'll get back to it later".

    Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

    If you disable DDMessageReader, do you still have an issue?

    I'm going to have to give that a shot. Thus far, I think the pattern seems to be that the header disappears when the message is marked as read by the recipient. I'm not 100% on that, though, I'll have to do some more digging.


    Well it's certainly possible to import messages without message-IDs. Perhaps if you ran "chksmb -e" you'd see a pattern with the messages that are missing message-IDs and identify the cause. In any case, it's not a fatal issue normally.

    here's my current chksmb -e output on dove-general and dove-syncops, the two echos that I've seen the issue crop up.

    https://pastebin.com/YacRr6XV

    After that, I tried a base fixsmb on dove-syncops, and here was the output:

    [root@bbs subs]# /sbbs/exec/fixsmb dove-syncops

    FIXSMB v2.10-Linux (rev 1.46) SMBLIB 2.60 - Rebuild Synchronet Message Base

    Opening dove-syncops
    99% #9698 (703F20)
    (704020) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7356448
    99%
    (704120) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7356704
    99% #9708 (705820)
    (705920) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7362848
    99%
    (705A20) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7363104
    100%
    Sorting index...
    Re-writing index...

    Saving message base status (9716 total messages).
    Closing message base.
    Done.

    DaiTengu

    ... Only dull people are brilliant at breakfast.

    ---
    Synchronet War Ensemble BBS - The sport is war, total war - warensemble.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Nightfox@1:103/705 to DaiTengu on Friday, March 29, 2019 12:59:54
    Re: Corrupt Message Headers
    By: DaiTengu to Digital Man on Fri Mar 29 2019 10:59 am

    Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

    I haven't seen that problem on my BBS with my reader, and haven't heard about this from other sysops using my reader either. The way it marks a message as read is by loading the message header without expanded fields (using MsgBase.get_msg_header()), applying the MSG_READ attribute, and then saving the header back to the messagebase (using MsgBase.put_msg_header()). I wouldn't expect that to corrupt the header.

    Nightfox

    ---
    Synchronet Digital Distortion: digitaldistortionbbs.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From DaiTengu@1:103/705 to Nightfox on Friday, March 29, 2019 16:30:05
    Re: Corrupt Message Headers
    By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

    I haven't seen that problem on my BBS with my reader, and haven't heard about this from other sysops using my reader either. The way it marks a message as read is by loading the message header without expanded fields (using MsgBase.get_msg_header()), applying the MSG_READ attribute, and then saving the header back to the messagebase (using MsgBase.put_msg_header()). I wouldn't expect that to corrupt the header.

    I wouldn't either, but right now it's the only hypothesis I have that fits my issues (header displays fine, once it's read, header disappears).

    I'm replying to this message just fine, the header was there, but it's the first time I viewed your message.

    EDIT: Yep, header disappeared when I went back and veiwed the messages in this area again.




    DaiTengu

    ... Ahh! Come on Erick, just this one last little feature!


    Mike

    ---
    Synchronet War Ensemble BBS - The sport is war, total war - warensemble.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Nightfox on Saturday, March 30, 2019 21:59:32
    Re: Corrupt Message Headers
    By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

    Re: Corrupt Message Headers
    By: DaiTengu to Digital Man on Fri Mar 29 2019 10:59 am

    Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

    I haven't seen that problem on my BBS with my reader, and haven't heard about this from other sysops using my reader either. The way it marks a message as read is by loading the message header without expanded fields (using MsgBase.get_msg_header()), applying the MSG_READ attribute, and then saving the header back to the messagebase (using MsgBase.put_msg_header()). I wouldn't expect that to corrupt the header.

    It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

    digital man

    This Is Spinal Tap quote #22:
    David St. Hubbins: Here lies David St. Hubbins... and why not?
    Norco, CA WX: 65.8F, 35.0% humidity, 0 mph SSE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to DaiTengu on Saturday, March 30, 2019 21:23:48
    Re: Corrupt Message Headers
    By: DaiTengu to Digital Man on Fri Mar 29 2019 10:59 am

    Re:
    By: to on Thu Mar 28 2019 01:02 am

    So, I read your reply to my message yesterday. It came up fine, with header information and everything. I didn't have time to pull the data, and thought to myself "I'll get back to it later".

    Today, when bringing up this message to reply to it, the message header was gone. Previous issues were in dove-general, and it appears to have spread to dove-syncops, now. I wonder if maybe DDMessageReader is doing something when marking the message as read? I'm not actually sure how that works.

    If you disable DDMessageReader, do you still have an issue?

    I'm going to have to give that a shot. Thus far, I think the pattern seems to be that the header disappears when the message is marked as read by the recipient. I'm not 100% on that, though, I'll have to do some more digging.


    Well it's certainly possible to import messages without message-IDs. Perhaps if you ran "chksmb -e" you'd see a pattern with the messages that are missing message-IDs and identify the cause. In any case, it's not a fatal issue normally.

    here's my current chksmb -e output on dove-general and dove-syncops, the two echos that I've seen the issue crop up.

    https://pastebin.com/YacRr6XV

    After that, I tried a base fixsmb on dove-syncops, and here was the output:

    [root@bbs subs]# /sbbs/exec/fixsmb dove-syncops

    FIXSMB v2.10-Linux (rev 1.46) SMBLIB 2.60 - Rebuild Synchronet Message Base

    Opening dove-syncops
    99% #9698 (703F20)
    (704020) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7356448
    99%
    (704120) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7356704
    99% #9708 (705820)
    (705920) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (2E 33 34 08) at offset 7362848
    99%
    (705A20) smb_getmsghdr returned -103:
    smb_getmsghdr corrupt message header ID (20 20 63 31) at offset 7363104
    100%
    Sorting index...
    Re-writing index...

    Saving message base status (9716 total messages).
    Closing message base.
    Done.

    And after that, does chksmb still report errors? If so, I recommend using smbutil to repair the msgbase:
    http://wiki.synchro.net/util:smbutil#examples

    digital man

    This Is Spinal Tap quote #37:
    David St. Hubbins: We are Spinal Tap from the UK - you must be the USA!
    Norco, CA WX: 65.0F, 40.0% humidity, 2 mph ESE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From DaiTengu@1:103/705 to Digital Man on Sunday, March 31, 2019 11:33:52
    Re: Corrupt Message Headers
    By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

    It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?


    Yep.

    Synchronet BBS for Linux Version 3.17 Debug

    Revision C Mar 25 2019 08:44 SMBLIB 2.60 GCC 4.8.5

    Copyright 2019 Rob Swindell - http://www.synchro.net

    JavaScript-C 1.8.5 2011-03-31

    GLIBC 2.17 cryptlib 3.4.4 (3440)

    Linux 4.18.16-x86_64-linode118 x86_64

    The problem started a week or so ago, I think? I noticed it's happening in the fidonet echos as well, as one of my users recently started posting there, and those messages are missing header information as well.

    Right now I'm reading/replying through the web interface as a lazy way to avoid disabling DDMsgReader globally.

    ---
    Synchronet War Ensemble BBS - The sport is war, total war - warensemble.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From DaiTengu@1:103/705 to Digital Man on Sunday, March 31, 2019 12:19:00
    Re: Corrupt Message Headers
    By: Digital Man to Nightfox on Sat Mar 30 2019 09:59 pm

    It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

    Viewing through the web interface seems to mark messages as "read", but the minute I went back and re-read them with DDMsgReader, the header information went away.

    I've now disabled DDMsgReader completely on my board, When the next message comes in addressed to me, I'll see if it happens with the stock Synchronet reader.

    DaiTengu

    ... The word 'meaningful' when used today is nearly always meaningless.

    ---
    Synchronet War Ensemble BBS - The sport is war, total war - warensemble.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to DaiTengu on Sunday, March 31, 2019 12:25:16
    Re: Corrupt Message Headers
    By: DaiTengu to Digital Man on Sun Mar 31 2019 11:33 am

    Re: Corrupt Message Headers
    By: Nightfox to DaiTengu on Fri Mar 29 2019 12:59 pm

    It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?


    Yep.

    Cool. I was asking Nightfox however, since he can't seem to reproduce the issue. Perhaps it was a recently-introduced bug.

    digital man

    This Is Spinal Tap quote #46:
    "Not an Exit" - we don't want an exit. Well that's true.
    Norco, CA WX: 79.0F, 19.0% humidity, 0 mph SSW wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Nightfox@1:103/705 to Digital Man on Sunday, March 31, 2019 17:21:04
    Re: Corrupt Message Headers
    By: Digital Man to Nightfox on Sat Mar 30 2019 09:59 pm

    I haven't seen that problem on my BBS with my reader, and haven't
    heard about this from other sysops using my reader either. The way it
    marks a message as read is by loading the message header without
    expanded fields (using MsgBase.get_msg_header()), applying the
    MSG_READ attribute, and then saving the header back to the messagebase
    (using MsgBase.put_msg_header()). I wouldn't expect that to corrupt
    the header.

    It's always possible I hosed something up in a recent commit to CVS. Are you running the latest and greatest?

    I wasn't, but just now I updated to the latest binaries (built today, 3/31). Now I'm seeing an error with my message reader - After reading a new message to me, my reader will add the MSG_READ attribute and save it back to the msgbase. With the latest binaries, I'm now seeing this error when calling MsgBase.put_msg_header():

    !JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529: Error: Error -110 adding SENDERNETADDR field to message header

    I wasn't seeing that error before. I'm not sure what would cause that error. My reader caches the message headers (it has an array of saved message headers) and it was applying the attribute to the cached header and then trying to save it to the msgbase. As a workaround, I tried the following:
    - Load the message header from the messagebase (without expanded fields)
    - Apply the new attribute (MSG_READ)
    - Save that header back to the msgbase
    However, on the first tep, when reading the header from the msgbase, it got a null object.

    Nightfox

    ---
    Synchronet Digital Distortion: digitaldistortionbbs.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Nightfox@1:103/705 to Digital Man on Sunday, March 31, 2019 17:36:48
    Re: Corrupt Message Headers
    By: Nightfox to Digital Man on Sun Mar 31 2019 05:21 pm

    I haven't seen that problem on my BBS with my reader, and haven't
    heard about this from other sysops using my reader either. The way
    it marks a message as read is by loading the message header without
    expanded fields (using MsgBase.get_msg_header()), applying the
    MSG_READ attribute, and then saving the header back to the
    messagebase (using MsgBase.put_msg_header()). I wouldn't expect that
    to corrupt the header.

    It's always possible I hosed something up in a recent commit to CVS.
    Are you running the latest and greatest?

    I wasn't, but just now I updated to the latest binaries (built today, 3/31). Now I'm seeing an error with my message reader - After reading a new message to me, my reader will add the MSG_READ attribute and save it back to the msgbase. With the latest binaries, I'm now seeing this error when calling MsgBase.put_msg_header():

    !JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529: Error: Error -110 adding SENDERNETADDR field to message header

    I wasn't seeing that error before. I'm not sure what would cause that

    Now that I've updated to today's binaries, I think I'm seeing the same issue as DaiTengu now. I read one of Vk3jed's messages to me, and after my reader
    tried to mark his message as read, now the header of the message from Vk3jed seems to be messed up. In my reader's reader interface, now it says it's written to 'All' instead of me, and the 'From' and 'Subject' are empty. In my reader's message list (where it displays the 'from', 'to', and subject, those are empty after my reader tried to apply the 'read' attribute to the header in the msgbase.

    Nightfox

    ---
    Synchronet Digital Distortion: digitaldistortionbbs.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From DaiTengu@1:103/705 to Nightfox on Monday, April 01, 2019 00:17:18
    Re: Corrupt Message Headers
    By: Nightfox to Digital Man on Sun Mar 31 2019 05:36 pm

    !JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529:
    Error: Error -110 adding SENDERNETADDR field to message header

    I saw that same error in my error log a couple days ago, but I haven't seen it since.

    Mar 29 16:28:37 bbs sbbs: term Node 1 <DaiTengu> !JavaScript ../xtrn/DDMsgReader/DDMsgReader.js line 1525: Error: Error -110 adding SENDERNETADDR field to message header

    DaiTengu

    ... Success is a public affair. Failure is a private funeral.

    ---
    Synchronet War Ensemble BBS - The sport is war, total war - warensemble.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Nightfox on Monday, April 01, 2019 16:15:38
    Re: Corrupt Message Headers
    By: Nightfox to Digital Man on Sun Mar 31 2019 05:36 pm

    Re: Corrupt Message Headers
    By: Nightfox to Digital Man on Sun Mar 31 2019 05:21 pm

    I haven't seen that problem on my BBS with my reader, and haven't
    heard about this from other sysops using my reader either. The way
    it marks a message as read is by loading the message header without
    expanded fields (using MsgBase.get_msg_header()), applying the
    MSG_READ attribute, and then saving the header back to the
    messagebase (using MsgBase.put_msg_header()). I wouldn't expect that
    to corrupt the header.

    It's always possible I hosed something up in a recent commit to CVS.
    Are you running the latest and greatest?

    I wasn't, but just now I updated to the latest binaries (built today, 3/31). Now I'm seeing an error with my message reader - After reading a new message to me, my reader will add the MSG_READ attribute and save it back to the msgbase. With the latest binaries, I'm now seeing this error when calling MsgBase.put_msg_header():

    !JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529: Error: Error -110 adding SENDERNETADDR field to message header

    I wasn't seeing that error before. I'm not sure what would cause that

    Now that I've updated to today's binaries, I think I'm seeing the same issue as DaiTengu now. I read one of Vk3jed's messages to me, and after my reader tried to mark his message as read, now the header of the message from Vk3jed seems to be messed up. In my reader's reader interface, now it says it's written to 'All' instead of me, and the 'From' and 'Subject' are empty. In my reader's message list (where it displays the 'from', 'to', and subject, those are empty after my reader tried to apply the 'read' attribute to the header in the msgbase.

    I reproduced and fixed the problem (in js_msgbase.c) - it was a bug introduced in the most recent change to the MsgBase.put_msg_header() implementation.


    digital man

    Synchronet "Real Fact" #74:
    Vertrauen went online (as a WWIV BBS running on a 10MHz PC-XT clone) in 1988. Norco, CA WX: 80.0F, 24.0% humidity, 10 mph NE wind, 0.00 inches rain/24hrs --- SBBSecho 3.07-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Nightfox@1:103/705 to Digital Man on Tuesday, April 02, 2019 09:29:28
    Re:
    By: to on Mon Apr 01 2019 04:15 pm

    !JavaScript ../xtrn/DigDist/MsgReader/DDMsgReader.js line 1529:
    Error: Error -110 adding SENDERNETADDR field to message header

    Now that I've updated to today's binaries, I think I'm seeing the
    same issue as DaiTengu now. I read one of Vk3jed's messages to me,

    I reproduced and fixed the problem (in js_msgbase.c) - it was a bug introduced in the most recent change to the MsgBase.put_msg_header() implementation.

    I updated to today's binaries, and the issue seems to be fixed now.

    Nightfox

    ---
    Synchronet Digital Distortion: digitaldistortionbbs.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)