Notepad++ cannot open a large text document. I would like to do this on a Linux system using a command in the terminal.
I believe
iconv might be what you're looking for. The following should do the trick:
iconv -f utf-8 -t utf-16le in_file.txt > out_file.txt
If you need to guess (as it's impossible to "determine" decisively) the encoding of a file, the following (on linux systems) may help:
file my_file.txt --mime-encoding
Or, if the situation calls for a more sophisticated tool, cchardet is pretty great and will even give you a "confidence level".
Here is a link to an article with some info about it (and character encodings in general) as well as some code you can use to get it to work.
Of course, if all else fails there's always running "split -b X my_file.txt" where X is some reasonable number of bytes (ie 2000000000 to 10000000000-ish), opening the resulting files in notepad++, doing whatever is needed there, and then cat-ing them back together afterwards.