mibuso.com

Microsoft Business Solutions online community
It is currently Wed May 22, 2013 6:52 pm

All times are UTC + 1 hour [ DST ]




Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Wed Mar 18, 2009 7:54 pm 
Offline

Joined: Fri Jan 23, 2004 4:57 pm
Posts: 29
Location: Cleveland, Oh
Country: United States (us)
I've recently seen a rash of situations where people are putting hidden characters into fields in NAV.
Specifically, they are getting Carriage Returns and Line Feeds into things like Invoice Numbers, Addresses, Names, etc...

We do a lot of extracting of data for analytical purposes, alot of which goes to flat files for exchange with corporate systems and warehouses.

When these files are created, they are screwed up because the bad data causes an premature break, adding extra lines and causing data exchange jobs to fail.

It's nearly impossible to actually type in a Line Feed or Carriage Return by hand, accidentally.

What we've found is that users are copying and pasting from websites (for addresses, etc...) and that is carrying along these hidden characters.

Has anyone else had this problem?


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Wed Mar 18, 2009 9:39 pm 
Offline
MVP Microsoft Dynamics NAV
Site Supporter

Joined: Fri Jun 06, 2003 9:01 pm
Posts: 6978
Location: L.I., New York
Country: United States (us)
How are you extracting data? with a dataport? perhaps you can detect & delete these characters as they come up?

Might not be the best way but you can check on entry of the field

"Your Field" - OnValidate()
Code: Select all
CHAR10 := 10;
CHAR13 := 13;
POS := STRPOS("Your Field", FORMAT(CHAR10));
IF POS <> 0 THEN ERROR('Pasting Canceled Carrage Return Exists');
POS := STRPOS("Your Field", FORMAT(CHAR13));
IF POS <> 0 THEN ERROR('Pasting Canceled Line Feed Exists');


Unfortunately sould like alot of fields they are pasting into. That's why I was thinking of cleaning it up on export. :-k

_________________
Harry Ruiz
http://www.cosmeticsolutions.com
http://www.autismspeaks.org


Top
 Profile E-mail WWW  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Thu Mar 19, 2009 10:20 am 
Offline

Joined: Mon Nov 29, 2004 2:51 pm
Posts: 288
Location: Slovenia
Country: Slovenia (si)
Change Codeunit:1 in function MakeText. I'm allowing pasting CR and LF in some fields that's why it is used only on text that is shorter or equal 50 chars (longer text is for composing additional descriptions on Items and similar)

Code: Select all
Local Vars
Name   DataType   Subtype   Length
cr   Char      
lf   Char      
tab   Char      

...
MakeText(VAR Text : Text[250]) : Integer
//AX001 - Start
Position := 1;
cr := 13 ;
lf := 10 ;
tab := 9 ;
Length := STRLEN(Text);
IF Length <= 50 THEN
  BEGIN
    Text := DELCHR(Text,'=',FORMAT(cr)) ;
    Text := DELCHR(Text,'=',FORMAT(lf)) ;
    Text := DELCHR(Text,'=',FORMAT(tab)) ;
  END ;
//AX001 - End
Length := STRLEN(Text);
ReadCharacter(' ',Text,Position,Length);
IF NOT ReadSymbol('?',Text,Position,Length) THEN
  EXIT(0);
...


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Thu Mar 19, 2009 10:32 am 
Offline
MVP Microsoft Dynamics NAV

Joined: Mon Dec 11, 2006 10:34 am
Posts: 2863
Location: Bergamo
Country: Italy (it)
Malajlo wrote:
Change Codeunit:1 in function MakeText. I'm allowing pasting CR and LF in some fields that's why it is used only on text that is shorter or equal 50 chars (longer text is for composing additional descriptions on Items and similar)

Code: Select all
Local Vars
Name   DataType   Subtype   Length
cr   Char      
lf   Char      
tab   Char      

...
MakeText(VAR Text : Text[250]) : Integer
//AX001 - Start
Position := 1;
cr := 13 ;
lf := 10 ;
tab := 9 ;
Length := STRLEN(Text);
IF Length <= 50 THEN
  BEGIN
    Text := DELCHR(Text,'=',FORMAT(cr)) ;
    Text := DELCHR(Text,'=',FORMAT(lf)) ;
    Text := DELCHR(Text,'=',FORMAT(tab)) ;
  END ;
//AX001 - End
Length := STRLEN(Text);
ReadCharacter(' ',Text,Position,Length);
IF NOT ReadSymbol('?',Text,Position,Length) THEN
  EXIT(0);
...

interesting.... :-k
another solution is: try with this(i've never used it, maybe you want to test it)
http://www.mibuso.com/dlinfo.asp?FileID=1011


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Apr 07, 2009 6:16 pm 
Offline

Joined: Fri Jan 23, 2004 4:57 pm
Posts: 29
Location: Cleveland, Oh
Country: United States (us)
The suggested code works beautifully for text fields (modifying the MakeText function).

BUT...it doesn't do anything for CODE fields.

And, the first problem I encountered was on the Vendor Invoice No. field, which is a CODE field.

Is there a similar function that handles input to all Code fields?


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Apr 07, 2009 6:39 pm 
Offline

Joined: Thu Mar 13, 2003 11:05 am
Posts: 161
Location: Italy
Country: Italy (it)
Hi

You can modify your personalization like this:

Code: Select all
...
MakeText(VAR Text : Text[250]) : Integer
//AX001 - Start
Position := 1;
Text2Remove[1] := 13;
Text2Remove[2] := 10;
Text2Remove[3] := 9;
IF STRLEN(Text) <= 50 THEN
  Text := DELCHR(Text,'=',Text2Remove) ;
//AX001 - End
Length := STRLEN(Text);
ReadCharacter(' ',Text,Position,Length);
IF NOT ReadSymbol('?',Text,Position,Length) THEN
  EXIT(0);
...


Bye

Matteo


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Apr 07, 2009 7:21 pm 
Offline

Joined: Sun Dec 10, 2006 9:21 am
Posts: 83
Location: Bayreuth
Country: Germany (de)
joshplummer wrote:
...BUT...it doesn't do anything for CODE fields...

Unfortunately the MakeText trigger in Codeunit 1 is only called automatically for fields of type Text (not for Code fields).


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Apr 07, 2009 7:44 pm 
Offline

Joined: Sun Dec 10, 2006 9:21 am
Posts: 83
Location: Bayreuth
Country: Germany (de)
joshplummer wrote:
...What we've found is that users are copying and pasting from websites

Well, the copy and paste problem for NAV field input is not restricted to copying from websites.
To see what I mean, just open Notepad and type "TEST" followed by a the enter key. Then select the whole text (with Ctrl+A) and paste it into a NAV text or code field... - you will see some extra characters (namely CR and LF) at the end of the string, when the input field in NAV is highlighted.
Since (as you already mentioned) it is nearly impossible to input CR/LF manually (not even with Alt+010 or Alt+013, etc.), I would concern this as a bug. There is a principle philosophy about copy and paste: It should not be possible to paste something that could not be entered manually or by means of a "legal" method.


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Apr 07, 2009 8:05 pm 
Offline

Joined: Fri Jan 23, 2004 4:57 pm
Posts: 29
Location: Cleveland, Oh
Country: United States (us)
gerdhuebner wrote:
joshplummer wrote:
...What we've found is that users are copying and pasting from websites

Well, the copy and paste problem for NAV field input is not restricted to copying from websites.
To see what I mean, just open Notepad and type "TEST" followed by a the enter key. Then select the whole text (with Ctrl+A) and paste it into a NAV text or code field... - you will see some extra characters (namely CR and LF) at the end of the string, when the input field in NAV is highlighted.
Since (as you already mentioned) it is nearly impossible to input CR/LF manually (not even with Alt+010 or Alt+013, etc.), I would concern this as a bug. There is a principle philosophy about copy and paste: It should not be possible to paste something that could not be entered manually or by means of a "legal" method.


Thanks. I understand the scope of the problem. Websites just happen to be where we are getting the majority of our issues.
In either case, the solution above fixes all the Text fields....but I still need a solution for the CODE fields.
Is there a way to intercept the function that applies the UPPERCASE to Code Fields? If so, I could add the appropriate code there....I assume not though.


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Apr 07, 2009 8:57 pm 
Offline
MVP Microsoft Dynamics NAV
Site Supporter

Joined: Fri Jun 06, 2003 9:01 pm
Posts: 6978
Location: L.I., New York
Country: United States (us)
So i guess you're looking for an "all over" solutions like modifying cu1. Is adding code to the problem code fields OnValidate too much work?

_________________
Harry Ruiz
http://www.cosmeticsolutions.com
http://www.autismspeaks.org


Top
 Profile E-mail WWW  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Wed Apr 08, 2009 9:03 am 
Offline
MVP Microsoft Dynamics NAV

Joined: Mon Dec 11, 2006 10:34 am
Posts: 2863
Location: Bergamo
Country: Italy (it)
Quote:
Is there a way to intercept the function that applies the UPPERCASE to Code Fields? If so, I could add the appropriate code there....I assume not though.

No, there is not such a function in nav...it's the datatype that uppercases the text...the "trigger" is the same that throws the error if you try to type 'a' in an integer field.


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Thu Apr 09, 2009 7:26 pm 
Offline

Joined: Sun Dec 10, 2006 9:21 am
Posts: 83
Location: Bayreuth
Country: Germany (de)
joshplummer wrote:
...Is there a way to intercept the function that applies the UPPERCASE to Code Fields?

Well, I think there might be some passable way to solve the problem... - In NAV there is a functionality called "Change Log". Every time records are changed, inserted or deleted some triggers in Codeunit 1 are called.
Here is a sample code snippet to prevent the user from modifying code fields with "illegally" pasted characters - provided you are not using the standard "Change Log" functionality (otherwise the code had to be modified...):
1.) Put the following code in the beginning of the method GetGlobalTableTriggerMask in Codeunit 1:
Code: Select all
EXIT(2);

This will cause NAV to call the method OnGlobalModify every time an arbitrary record is modified by the user.
2.) Put the following code in the beginning of the method OnGlobalModify in Codeunit 1:
Code: Select all
illegalString[1] := 13;
illegalString[2] := 10;
// ...
Field.SETRANGE(TableNo,RecRef.NUMBER);
Field.SETRANGE(Type,Field.Type::Code);
Field.SETRANGE(Class,Field.Class::Normal);
Field.SETRANGE(Enabled,TRUE);
IF Field.FIND('-') THEN
  REPEAT
    fRef := RecRef.FIELD(Field."No.");
    IF DELCHR(FORMAT(fRef),'=',illegalString) <> FORMAT(fRef) THEN
      ERROR('Illegal characters in field "%1".',Field."Field Caption");
  UNTIL Field.NEXT = 0;

Here, illegalString is a text variable of suitable length, containg the "illegal" characters you want to test. Field is a record variable for the Field-Table (virtual table no. 2000000041), fRef is a FieldRef variable.
Be sure to restart your NAV client after modifying Codeunit 1 for changes to take effect.

One disadvantage of the above method (besides evtl. decrease in performance) is that the error is thrown not until the record is to be modified, causing all evtl. inputs in other fields to be lost. The advantage of course is that you can check all code fields in all tables with little coding effort. The method can be extended (if necessary) for insert and/or rename transactions. Instead of throwing an error it is also possible to remove the illegal characters automatically (without notice for the user). This can be achieved with the following modification:
Code: Select all
illegalString[1] := 13;
illegalString[2] := 10;
// ...
Field.SETRANGE(TableNo,RecRef.NUMBER);
Field.SETRANGE(Type,Field.Type::Code);
Field.SETRANGE(Class,Field.Class::Normal);
Field.SETRANGE(Enabled,TRUE);
IF Field.FIND('-') THEN
  REPEAT
    fRef := RecRef.FIELD(Field."No.");
    fRef.VALUE := DELCHR(FORMAT(fRef),'=',illegalString);
  UNTIL Field.NEXT = 0;


Top
 Profile  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Aug 03, 2010 2:08 pm 
Offline

Joined: Tue Dec 08, 2009 11:40 am
Posts: 43
Country: Netherlands (nl)
Very late reply, but you can copy the MakeText trigger, rename it MakeCode and give it ID 109.

100 MakeSpecial (DateFormula, RecordID, TableFilter)
200 MakeSpecialFilter
101 MakeBoolean
201 MakeBooleanFilter
102 MakeOption
202 MakeOptionFilter
103 MakeInteger
203 MakeIntegerFilter
104 MakeDecimal
204 MakeDecimalFilter
105 MakeDate
205 MakeDateFilter
106 MakeTime
206 MakeTimeFilter
107 MakeText
207 MakeTextFilter
109 MakeCode
209 MakeCodeFilter
110 MakeBinary
210 MakeBinaryFilter
111 MakeBLOB
211 MakeBLOBFilter
112 MakeBigInteger
212 MakeBigIntegerFilter
113 MakeDuration
213 MakeDurationFilter
114 MakeDateTime
214 MakeDateTimeFilter
115 MakeGUID
215 MakeGUIDFilter

Did not test them all but just know the IDs when they are called.


Top
 Profile E-mail WWW  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Tue Aug 10, 2010 6:50 pm 
Offline

Joined: Mon Mar 12, 2007 3:07 pm
Posts: 340
Location: Hamburg
Country: Germany (de)
=D> =D> =D> =D> =D> =D> =D> =D> =D> =D>
You are the Best! :thumbsup:

In Fu MakeText I added the below code to remove CR/LFs ages ago. Now I added Fu MakeCode (ID 109) with the same code and there are no more CR/LFs in the DB :D \:D/
Code: Select all
CharToDelete_l[1] := 10;
CharToDelete_l[2] := 13;
CharToDelete_l[3] := 9;

CodeText := DELCHR(CodeText, '=', CharToDelete_l);

Another good reason to call all the old customers again and make them a little happier.

_________________
Frank Dickschat
FD Consulting


Top
 Profile E-mail WWW  
 
 Post subject: Re: Hidden Characters (Carriage Return and Line Feed)
PostPosted: Wed Aug 10, 2011 11:26 pm 
Offline

Joined: Thu Jul 21, 2011 3:19 pm
Posts: 5
Country: United States (us)
Duikmeester wrote:
Very late reply, but you can copy the MakeText trigger, rename it MakeCode and give it ID 109.

100 MakeSpecial (DateFormula, RecordID, TableFilter)
200 MakeSpecialFilter
101 MakeBoolean
201 MakeBooleanFilter
102 MakeOption
202 MakeOptionFilter
103 MakeInteger
203 MakeIntegerFilter
104 MakeDecimal
204 MakeDecimalFilter
105 MakeDate
205 MakeDateFilter
106 MakeTime
206 MakeTimeFilter
107 MakeText
207 MakeTextFilter
109 MakeCode
209 MakeCodeFilter
110 MakeBinary
210 MakeBinaryFilter
111 MakeBLOB
211 MakeBLOBFilter
112 MakeBigInteger
212 MakeBigIntegerFilter
113 MakeDuration
213 MakeDurationFilter
114 MakeDateTime
214 MakeDateTimeFilter
115 MakeGUID
215 MakeGUIDFilter

Did not test them all but just know the IDs when they are called.



Where do I specify the ID Number for the new function? Where are the ID Number's for Functions located in general. Sorry if this is basic. Thanks.


Top
 Profile E-mail  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next

All times are UTC + 1 hour [ DST ]


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum


Search for:
Jump to: