Can't read unicode file created by PB

Just starting out? Need help? Post your questions and find answers here.
morosh
Posts: 27
Joined: Mon Feb 02, 2015 7:48 pm

Can't read unicode file created by PB

Post by morosh »

Hello:
I need to read a unicode file created by PB, like:

Code: Select all

OpenFile(0,"test.txt", #PB_Unicode)
WriteByte(0,255)    ; signature
WriteByte(0,254)    ; signature
WriteStringN(0,"ABCDEF", #PB_Unicode)
WriteStringN(0,"XYZTUV", #PB_Unicode)
WriteStringN(0,"KLMNOPQ", #PB_Unicode)
WriteStringN(0,"0123456", #PB_Unicode)
WriteStringN(0,"789", #PB_Unicode)
WriteStringN(0,"GHIJKL", #PB_Unicode)
CloseFile(0)
Then I tried to read it in SB, the first line is read OK, after there is chinese characters with infinite loop:

Code: Select all

Procedure ReadCallback(Status, Filename$, File, Size)
  If Status = #PB_Status_Loaded
    Debug "File: " + Filename$ + " - Size: " + Size + " bytes"

    While Not Eof(0)
      Debug Str(NbLine) + ":"+ReadString(0,#PB_Unicode)+"*"
      NbLine+1
      If NbLine > 20
        Break
      EndIf
    Wend
    CloseFile(0)
  ElseIf Status = #PB_Status_Error
    Debug "Error when loading the file: " + Filename$
  EndIf
EndProcedure
   
OpenFile(0, "test.txt", @ReadCallback(), #PB_Unicode)
if the same file is created in SB, everything is OK.
After investigation, I noticed that the line'termination in PB is: 0D 00 0A 00, but in SB it's: 0A 00
if I removed manually (with a hex editor) the extra 0D 00, the file is read correctly.

is this a bug?

Thanks in advance
kpeters58
Posts: 19
Joined: Fri Dec 07, 2018 7:27 pm
Location: BC, Canada

Re: Can't read unicode file created by PB

Post by kpeters58 »

Well, it appears to be the age old issue with CR/LF vs LF line termination.

Be aware, that if you use GIT (or some other version control?) that CVS systems sometimes wreak havoc on line termination as well if not configured properly.

It should be trivial to add support for both formats.
Fred's already done it for PB, where you can choose which line terminator(s) you want under Compiler Settings.
morosh
Posts: 27
Joined: Mon Feb 02, 2015 7:48 pm

Re: Can't read unicode file created by PB

Post by morosh »

Thank you
Meanwhile, I'll change my PB program to put explicitly 0D 00 as a line termination
Post Reply