Delphi Pages Forums

Delphi Pages Forums (http://www.delphipages.com/forum/index.php)
-   VCL (http://www.delphipages.com/forum/forumdisplay.php?f=4)
-   -   Read file above 3.5GB (http://www.delphipages.com/forum/showthread.php?t=217778)

oxygen 06-30-2018 09:23 AM

Read file above 3.5GB
 
Hello, can you help me? I need to read file which is ~3.7GB in size and save it to smaller files with size ~300kB.

Using Delphi 7. RAM limit cca 2600MB.

rojam 07-02-2018 01:16 AM

Quote:

Originally Posted by oxygen (Post 234210)
Hello, can you help me? I need to read file which is ~3.7GB in size and save it to smaller files with size ~300GB.

Using Delphi 7. RAM limit cca 2600MB.

I believe that D7 can handle files that large using TFileStream. To ensure, Check the definition of a TFileStream property such as Position and ensure it is defined as an Int64. (I don't have D7 installed to check myself)
As an example of how to break up a file, I wrote two procedures that split a file in half and one that rejoins them.

The two halves are renamed with a part1 and part2 extension.

Code:

TForm1 = class(TForm)
  ..
private
  procedure SplitFile(const TheFile: String);//add these two methods
  procedure ReJoinFile(const InFileName, OutFileName: String);
end;

.....

procedure TForm1.SplitFile(const TheFile: String);
var
  HalfSize: Int64;
  FullSize: Int64;
  InFS: TFileStream;
  OutFS: TFileStream;
  I: Integer;
  FileName: String;
  FileWOExt: String;
  FilePath: String;
  OutFile: String;
begin
  FilePath := ExtractFilePath(TheFile);
  FileName := ExtractFileName(TheFile);
  FileWOExt := ChangeFileExt(FileName, '');
  I := 1;
  InFS := TFileStream.Create(TheFile, fmOpenRead);
  try
    FullSize := FileSize(TheFile);
    HalfSize := FullSize div 2;
    while I <= 2 do
    begin
      OutFile := FilePath + FileWOExt + '.Part' + IntToStr(I);
      OutFS := TFileStream.Create(OutFile, fmCreate);
      try
        if I = 1 then
          OutFS.CopyFrom(InFS, HalfSize)
        else
          OutFS.CopyFrom(InFS, InFS.Size - InFS.Position);
        Inc(I);
      finally
        OutFS.Free;
      end;
    end;
  finally
    InFS.Free;
  end;
end;

procedure TForm1.ReJoinFile(const InFileName, OutFileName: String);
var
  I: Integer;
  InFS: TFileStream;
  OutFS: TFileStream;
begin
  I := 1;
  OutFS := TFileStream.Create(OutFileName, fmCreate);
  try
    while I <= 2 do
    begin
      InFS := TFileStream.Create(InFileName + '.part' + IntToStr(I), fmOpenRead);
      try
        OutFS.CopyFrom(InFS, InFS.Size);
        Inc(I);
      finally
        InFS.Free;
      end;
    end;
  finally
    OutFS.Free;
  end;
end;

to use them

Code:

procedure TForm1.Button1Click(Sender: TObject);
begin
  SplitFile('c:\Test\YourFileName.ext);
  //param1 is the path/filename of the file to split
end;

procedure TForm1.Button2Click(Sender: TObject);
begin
  ReJoinFile('c:\Test\YourFileName', 'c:\Test\YourNewFile.ext');
  //param1 is the path/filename without extension since it will look for part1 and part2
  //param2 is the new file name that it will create after joining the 2 halves
end;

Also, the application will need to have write access to the directory it's going to write the split files and rejoined file.

oxygen 07-02-2018 07:42 AM

Thank you for your code. I mean 300KB chunks not 300GB. I am using Wiondows XP 32 bit.

rojam 07-02-2018 08:11 PM

the split code would be something like this (untested)
Code:

procedure TForm1.SplitFile(const TheFile: String);
var
  maxSplitPart: Integer;
  InFS: TFileStream;
  OutFS: TFileStream;
  I: Integer;
  FileName: String;
  FileWOExt: String;
  FilePath: String;
  OutFile: String;
begin
  maxPartSize := 300000; //300 kb in bytes
  FilePath := ExtractFilePath(TheFile);
  FileName := ExtractFileName(TheFile);
  FileWOExt := ChangeFileExt(FileName, '');
  I := 1;
  Done := False;
  While Not Done do
  begin
    InFS := TFileStream.Create(TheFile, fmOpenRead);
    try
      OutFile := FilePath + FileWOExt + '.Part' + IntToStr(I);
      Inc(I);
      OutFS := TFileStream.Create(OutFile, fmCreate);
      try
        if maxPartSize < (InFS.Size - InFS.Position)
          OutFS.CopyFrom(InFS, maxSplitPart)
        else
        begin
          OutFS.CopyFrom(InFS, InFS.Size - InFS.Position);
          Done := True;
        end;
      finally
        OutFS.Free;
      end;
    finally
      InFS.Free;
    end;
  end;
end;


oxygen 07-06-2018 11:23 AM

Thank you for your code. I watched it and used similar solution based on file streaming.


All times are GMT. The time now is 11:30 AM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2019, vBulletin Solutions, Inc.