Large TXT files
- Thread starter nitko12
- Start date
For what purpose?
Most text editors load a whole file in memory -- and the size is either limited by the editor's code, or your system RAM.
You can write your own C code to extract the characters sequentially from the file for viewing (i.e print to the display).
But I cannot think of a single instance where, or why, one would want to view (and potentially edit) 50GB of text data at one time.
I dont understand a word you said but is there any program to open it simply by clicking on it because i dont know a thing about c codes or anything based on programing
You still haven't said what you want to do with the data. We are talking 53,687,091,200 characters. Do you just want to look at them (do you have enough days left in your life?), do you want to select a reasonable subset of them, or do you want to process them somehow?
Processing a text file this size is not trivial.
Very roughly speaking, 5000 characters in a text file is approximately 5kB (4.88kB to be more precise). There are 1,048,576 kB in one GB. That means that in your 50GB text file there is more than 52,000,000,000 characters. How many of those do you really need?
EDIT: @joeyd999 you beat me to it!
EDIT: @joeyd999 you beat me to it!
Linux/Unix also has a 'split' command:
split --bytes=1M /path/to/file/textfile.txt /path/to/file/prefixForfiles
This will split the large file into 50,000 individual 1 megabyte files that you could then individually view with a capable text editor. Some editors are limited to < 64K bytes, so:
split --bytes=50K /path/to/file/textfile.txt /path/to/file/prefixForfiles
would give you 1,000,000 individual 50,000 byte files.
Have fun.
split --bytes=1M /path/to/file/textfile.txt /path/to/file/prefixForfiles
This will split the large file into 50,000 individual 1 megabyte files that you could then individually view with a capable text editor. Some editors are limited to < 64K bytes, so:
split --bytes=50K /path/to/file/textfile.txt /path/to/file/prefixForfiles
would give you 1,000,000 individual 50,000 byte files.
Have fun.
He could send them to a random overseas fax......Just don't try printing it out.
Not one that I know of or can verify will work. Until recently we never even had hard drives that large, let alone a text file that big. My best guess is that Notepad++ might be able to do the job.
https://notepad-plus-plus.org/
https://notepad-plus-plus.org/
Is the data one line with 50G characters?
If not, 'sed' was designed to operate on arbitrarily large files one line at a time.
If it's one very long line, you could try 'dd' and specify a number of bytes to read at a time. Subsequent blocks of data would require a byte offset to be specified. It could be very slow without preprocessing the data. You could use 'dd' to partition the data into files of more manageable size...
