Delphi Pages Forums  

Go Back   Delphi Pages Forums > Delphi Forum > General

Lost Password?

Reply
 
Thread Tools Display Modes
  #1  
Old 10-18-2008, 03:54 AM
BaraoZemo BaraoZemo is offline
Senior Member
 
Join Date: Nov 2001
Posts: 3,598
Default count the number os lines of a text file

hi,


i have a large CSV file... (15 mb)

but i need a fast function that return to me the number of lines of the CSV file.

i need to know the "number of lines" of the csv file, because i'm will load it inside a stringgrid, and stringgrd need that i put the exact number of lines that the file have (before open it inside the grid)...

tks



Regards
BaraoZemo

ps, don't forget to click in the button and Accept as Answer if this helped you!
Reply With Quote
  #2  
Old 10-18-2008, 04:09 AM
Jasser Jasser is offline
Moderator
 
Join Date: Jan 2005
Location: Saudi Arabia
Posts: 5,059
Default RE: count the number os lines of a text file

I like TStringlist object. You can use it to get the number of lines and also to read the CSV file and put it in the string grid.


[DELPHI]var
i : Integer;
oSL : TStringlist;
begin
oSL := TStringlist.Create;
oSL.LoadFromFile('C:\TheFile.CSV');

Showmessage('Lines count are' + IntToStr(oSL.Count));

//you can fill the stringgrid from oSL here.
for i:=0 to oSL.Count -1 do begin
StringGrid1.Cells[0,i+1] := Copy(oSL[i],1,20); //Read the first cell data.
StringGrid1.Cells[0,i+1] := Copy(oSL[i],21,10); //Read the second cell data.
end;


oSL.Free;[/DELPHI]

Regards,
Abdulaziz Jasser
Reply With Quote
  #3  
Old 10-18-2008, 04:13 AM
chris_w chris_w is offline
Senior Member
 
Join Date: Jan 2004
Posts: 1,397
Default RE: count the number os lines of a text file

[pre]
Read chunks of the file into a string or an array of byte and
count the number of linefeed characters ( #10 ). Add 1 to the
total if the last line does not end with a linefeed character.
64K is usually a pretty good size for the buffer.
[/pre]
[pre]
"There is a theory which states that if ever anybody discovers
exactly what the Universe is for and why it is here, it will
instantly disappear and be replaced by something even more
bizarre and inexplicable. There is another theory which states
that this has already happened."
-- Douglas Adams
[/pre]

Chris
Reply With Quote
  #4  
Old 10-18-2008, 07:08 AM
BaraoZemo BaraoZemo is offline
Senior Member
 
Join Date: Nov 2001
Posts: 3,598
Default RE: count the number os lines of a text file

hi Jasser,

I was with the my head hollow.

but I ended tends to stop and to do the routine for right here...

Code:
function FastLinesCount(sfilename:string):extended;
var
 hFile : TextFile;
 sLine : String;
 iLinescount: Extended;
begin
result:=0;
if not FileExists(sfilename) then exit;

AssignFile(hFile, sFileName);
Reset(hFile);

iLinescount:=0;
while NOT EOF(hFile) do
 begin
   ReadLn(hFile, sLine);
   iLinescount:=iLinescount+1;
 end;

result:=iLinescount;
end;


procedure TForm1.Button1Click(Sender: TObject);
begin
ShowMessage(FloatToStr(FastLinesCount(Edit1.text)));
end;

Regards
BaraoZemo

ps, don't forget to click in the button and Accept as Answer if this helped you!
Reply With Quote
  #5  
Old 10-18-2008, 07:22 AM
chris_w chris_w is offline
Senior Member
 
Join Date: Jan 2004
Posts: 1,397
Default RE: count the number os lines of a text file

Using ReadLn(hFile) without the string var should be faster.

Chris
Reply With Quote
  #6  
Old 10-18-2008, 09:22 AM
Jasser Jasser is offline
Moderator
 
Join Date: Jan 2005
Location: Saudi Arabia
Posts: 5,059
Default RE: count the number os lines of a text file

BaraoZemo,

I've done some benchmarking and I found using TStringlist with file that is 15MB is faster than your code. Try it your self. My function is bellow.

[DELPHI]function GetLinesCount(sFileName : String): Integer;
var
i : Integer;
oSL : TStringlist;
begin
oSL := TStringlist.Create;
oSL.LoadFromFile(sFileName);

result := oSL.Count;

oSL.Free;
end;

//This is the way I tested the code:

var
iStart : Int64;
begin
iStart := GetTickCount;

GetLinesCount(Edit1.text);
// FastLinesCount(Edit1.text);

self.Caption := Format('%d',[GetTickCount-iStart]);[/DELPHI]

Regards,
Abdulaziz Jasser
Reply With Quote
  #7  
Old 10-18-2008, 09:28 AM
Jasser Jasser is offline
Moderator
 
Join Date: Jan 2005
Location: Saudi Arabia
Posts: 5,059
Default RE: count the number os lines of a text file

I removed the "i" variable declaration. Now it's even smaller:

[DELPHI]function GetLinesCount(sFileName : String): Integer;
var
oSL : TStringlist;
begin
oSL := TStringlist.Create;
oSL.LoadFromFile(sFileName);

result := oSL.Count;

oSL.Free;
end;[/DELPHI]

Regards,
Abdulaziz Jasser
Reply With Quote
  #8  
Old 10-18-2008, 11:26 AM
chris_w chris_w is offline
Senior Member
 
Join Date: Jan 2004
Posts: 1,397
Default RE: count the number os lines of a text file

[pre]
Smaller yes, fastest no

[/pre][DELPHI]
//used to create a 15MB test file
procedure TForm1.Button1Click(Sender: TObject);
var
f, s : string;
strs : TStringList;
i : integer;
fs : TFileStream;
b : byte;
begin
f := IncludeTrailingPathDelimiter(ExtractFilePath(Param Str(0))) +'15MB_TestFile.txt';

if FileExists(f) then
if MessageBox(Handle,
'File exists, create new anyway?',
'Confirm',
MB_ICONQUESTION OR MB_YESNO) <> IDYES then exit;

strs := TStringList.Create;
Screen.Cursor := crHourGlass;
try
s := f +f +f;
for i := 0 to 15000000 div Length(s) do
strs.Add(s);

strs.SaveToFile(f);
finally
strs.Free;
Screen.Cursor := crDefault;
end;

(*
//add Ctrl+Z character
fs := TFileStream.Create(f, fmOpenReadWrite);
try
fs.Seek(0, soFromEnd);
b := 26; //Ctrl +Z
fs.Write(b, 1);
finally
fs.Free;
end;
*)
end;

function LnCount(const Filename: string): integer;
var
FS : TFileStream;
bytes : array[0..$10000]of byte;
i, red : integer;
begin
result := 0;
if FileExists(Filename) then begin
FS := TFileStream.Create(Filename, fmOpenRead or fmShareDenyNone);
try
red := FS.Read(bytes[0], $10000);
while red > 0 do begin
for i := 0 to red -1 do
if bytes[i] = 10 then
Inc(result);

red := FS.Read(bytes[0], $10000);
end;

if FS.Size > 0 then begin
//see if last line ends with a linefeed character
if FS.Size >= $10000 then
FS.Seek(-$10000, soFromEnd)
else
FS.Position := 0;

red := FS.Read(bytes[0], $10000);
i := red -1;
//skip bytes < 9 or equal to Ctrl+Z (26)
while (i > -1) and ((bytes[i] < 9) or (bytes[i] = 26)) do
Dec(i);

if (i > -1) and (bytes[i] <> 10) then
Inc(result);
end;

finally
FS.Free;
end;
end;
end;

function FastLinesCount(sfilename:string):extended;
var
hFile : TextFile;
iLinescount: Extended;
begin
result:=0;
if not FileExists(sfilename) then exit;

AssignFile(hFile, sFileName);
Reset(hFile);

iLinescount:=0;
while NOT EOF(hFile) do
begin
ReadLn(hFile);
iLinescount:=iLinescount+1;
end;

result:=iLinescount;
end;

function GetLinesCount(sFileName : String): Integer;
var
oSL : TStringlist;
begin
oSL := TStringlist.Create;
oSL.LoadFromFile(sFileName);

result := oSL.Count;

oSL.Free;
end;



procedure TForm1.Button2Click(Sender: TObject);
var
s, f, a, b, c : cardinal;
ia, ib, ic : integer;
fn : string;
begin
fn := IncludeTrailingPathDelimiter(ExtractFilePath(Param Str(0))) +'15MB_TestFile.txt';

if not FileExists(fn) then begin
MessageBox(Handle,
'File does not exist.',
'Error',
MB_ICONSTOP OR MB_OK);
exit;
end;

Screen.Cursor := crHourGlass;
try
s := GetTickCount();
ia := LnCount(fn);
f := GetTickCount();
a := f -s;

s := GetTickCount();
ib := Round(FastLinesCount(fn));
f := GetTickCount();
b := f -s;

s := GetTickCount();
ic := GetLinesCount(fn);
f := GetTickCount();
c := f -s;

Label1.Caption := Format('Lines: %.n Ticks: %.n', [ia *1.0, a *1.0]);
Label2.Caption := Format('Lines: %.n Ticks: %.n', [ib *1.0, b *1.0]);
Label3.Caption := Format('Lines: %.n Ticks: %.n', [ic *1.0, c *1.0]);
finally
Screen.Cursor := crDefault;
end;
end;

[/DELPHI][pre]
"There is a theory which states that if ever anybody discovers
exactly what the Universe is for and why it is here, it will
instantly disappear and be replaced by something even more
bizarre and inexplicable. There is another theory which states
that this has already happened."
-- Douglas Adams
[/pre]

Chris
Reply With Quote
  #9  
Old 10-19-2008, 04:44 AM
BaraoZemo BaraoZemo is offline
Senior Member
 
Join Date: Nov 2001
Posts: 3,598
Default RE: count the number os lines of a text file

tks all...

jasser, chris...

this is the result from the "real csv file"
(not the csv generated automatically)

19/10/2008 10:47:42
lncount: Lines: 335.340 Ticks: 46
fastlinecount: Lines: 335.340 Ticks: 235
getlinescount: Lines: 335.340 Ticks: 250

19/10/2008 10:47:44
lncount: Lines: 335.340 Ticks: 62
fastlinecount: Lines: 335.340 Ticks: 234
getlinescount: Lines: 335.340 Ticks: 235

19/10/2008 10:47:47
lncount: Lines: 335.340 Ticks: 63
fastlinecount: Lines: 335.340 Ticks: 234
getlinescount: Lines: 335.340 Ticks: 250

19/10/2008 10:47:52
lncount: Lines: 335.340 Ticks: 47
fastlinecount: Lines: 335.340 Ticks: 234
getlinescount: Lines: 335.340 Ticks: 250

19/10/2008 10:47:54
lncount: Lines: 335.340 Ticks: 47
fastlinecount: Lines: 335.340 Ticks: 234
getlinescount: Lines: 335.340 Ticks: 250

19/10/2008 10:48:12
lncount: Lines: 335.340 Ticks: 46
fastlinecount: Lines: 335.340 Ticks: 235
getlinescount: Lines: 335.340 Ticks: 250

Regards
BaraoZemo

ps, don't forget to click in the button and Accept as Answer if this helped you!
Reply With Quote
  #10  
Old 10-19-2008, 07:59 AM
Jasser Jasser is offline
Moderator
 
Join Date: Jan 2005
Location: Saudi Arabia
Posts: 5,059
Default RE: count the number os lines of a text file

With streams is always faster. But the test that I made show TStringlist is faster the code you posted before. Any how, it is better to use streams for peformnace.

Regards,
Abdulaziz Jasser
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is On

Forum Jump


All times are GMT. The time now is 12:10 AM.


Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2019, vBulletin Solutions, Inc.