Delphi Pages Forums  

Go Back   Delphi Pages Forums > Delphi Forum > VCL

Lost Password?

Reply
 
Thread Tools Display Modes
  #1  
Old 06-30-2018, 09:23 AM
oxygen oxygen is offline
Junior Member
 
Join Date: May 2018
Posts: 5
Default Read file above 3.5GB

Hello, can you help me? I need to read file which is ~3.7GB in size and save it to smaller files with size ~300kB.

Using Delphi 7. RAM limit cca 2600MB.

Last edited by oxygen; 07-02-2018 at 07:39 AM.
Reply With Quote
  #2  
Old 07-02-2018, 01:16 AM
rojam rojam is offline
Senior Member
 
Join Date: Jun 2015
Posts: 183
Default

Quote:
Originally Posted by oxygen View Post
Hello, can you help me? I need to read file which is ~3.7GB in size and save it to smaller files with size ~300GB.

Using Delphi 7. RAM limit cca 2600MB.
I believe that D7 can handle files that large using TFileStream. To ensure, Check the definition of a TFileStream property such as Position and ensure it is defined as an Int64. (I don't have D7 installed to check myself)
As an example of how to break up a file, I wrote two procedures that split a file in half and one that rejoins them.

The two halves are renamed with a part1 and part2 extension.

Code:
TForm1 = class(TForm)
  ..
private
  procedure SplitFile(const TheFile: String);//add these two methods
  procedure ReJoinFile(const InFileName, OutFileName: String);
end;

.....

procedure TForm1.SplitFile(const TheFile: String);
var
  HalfSize: Int64;
  FullSize: Int64;
  InFS: TFileStream;
  OutFS: TFileStream;
  I: Integer;
  FileName: String;
  FileWOExt: String;
  FilePath: String;
  OutFile: String;
begin
  FilePath := ExtractFilePath(TheFile);
  FileName := ExtractFileName(TheFile);
  FileWOExt := ChangeFileExt(FileName, '');
  I := 1;
  InFS := TFileStream.Create(TheFile, fmOpenRead);
  try
    FullSize := FileSize(TheFile);
    HalfSize := FullSize div 2;
    while I <= 2 do
    begin
      OutFile := FilePath + FileWOExt + '.Part' + IntToStr(I);
      OutFS := TFileStream.Create(OutFile, fmCreate);
      try
        if I = 1 then
          OutFS.CopyFrom(InFS, HalfSize)
        else
          OutFS.CopyFrom(InFS, InFS.Size - InFS.Position);
        Inc(I);
      finally
        OutFS.Free;
      end;
    end;
  finally
    InFS.Free;
  end;
end;

procedure TForm1.ReJoinFile(const InFileName, OutFileName: String);
var
  I: Integer;
  InFS: TFileStream;
  OutFS: TFileStream;
begin
  I := 1;
  OutFS := TFileStream.Create(OutFileName, fmCreate);
  try
    while I <= 2 do
    begin
      InFS := TFileStream.Create(InFileName + '.part' + IntToStr(I), fmOpenRead);
      try
        OutFS.CopyFrom(InFS, InFS.Size);
        Inc(I);
      finally
        InFS.Free;
      end;
    end;
  finally
    OutFS.Free;
  end;
end;
to use them

Code:
procedure TForm1.Button1Click(Sender: TObject);
begin
  SplitFile('c:\Test\YourFileName.ext);
  //param1 is the path/filename of the file to split
end;

procedure TForm1.Button2Click(Sender: TObject);
begin
  ReJoinFile('c:\Test\YourFileName', 'c:\Test\YourNewFile.ext');
  //param1 is the path/filename without extension since it will look for part1 and part2
  //param2 is the new file name that it will create after joining the 2 halves
end;
Also, the application will need to have write access to the directory it's going to write the split files and rejoined file.
Reply With Quote
  #3  
Old 07-02-2018, 07:42 AM
oxygen oxygen is offline
Junior Member
 
Join Date: May 2018
Posts: 5
Default

Thank you for your code. I mean 300KB chunks not 300GB. I am using Wiondows XP 32 bit.
Reply With Quote
  #4  
Old 07-02-2018, 08:11 PM
rojam rojam is offline
Senior Member
 
Join Date: Jun 2015
Posts: 183
Default

the split code would be something like this (untested)
Code:
procedure TForm1.SplitFile(const TheFile: String);
var
  maxSplitPart: Integer;
  InFS: TFileStream;
  OutFS: TFileStream;
  I: Integer;
  FileName: String;
  FileWOExt: String;
  FilePath: String;
  OutFile: String;
begin
  maxPartSize := 300000; //300 kb in bytes
  FilePath := ExtractFilePath(TheFile);
  FileName := ExtractFileName(TheFile);
  FileWOExt := ChangeFileExt(FileName, '');
  I := 1;
  Done := False;
  While Not Done do
  begin
    InFS := TFileStream.Create(TheFile, fmOpenRead);
    try
      OutFile := FilePath + FileWOExt + '.Part' + IntToStr(I);
      Inc(I);
      OutFS := TFileStream.Create(OutFile, fmCreate);
      try
        if maxPartSize < (InFS.Size - InFS.Position)
          OutFS.CopyFrom(InFS, maxSplitPart)
        else
        begin
          OutFS.CopyFrom(InFS, InFS.Size - InFS.Position);
          Done := True;
        end;
      finally
        OutFS.Free;
      end;
    finally
      InFS.Free;
    end;
  end;
end;
Reply With Quote
  #5  
Old 07-06-2018, 11:23 AM
oxygen oxygen is offline
Junior Member
 
Join Date: May 2018
Posts: 5
Default

Thank you for your code. I watched it and used similar solution based on file streaming.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 08:30 AM.


Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2018, vBulletin Solutions, Inc.