Awesome
DiscordChatExporter-frontend (DCEF / DCE-Frontend)
View and search your DiscordChatExporter exports in a Discord like user interface. Try a demo in your browser :)
Features
- Familiar Discord-like interface
- Search with autocomplete
- Optimized to handle really large exports well
- You own the data - use locally or self-host on a server
- Runs on Linux or Windows
Want to chat? Join Tyrrrz's discord server. I usually hang out in
#dce-frontend
channel.
NOTE: The exporter DiscordChatExporter is maintained by different maintainer than this viewer.
How to use
1. Check prerequisites
Windows:
- Windows 10 or newer
- x64 architecture
- CPU with AVX support
- Available (and not excluded) ports
21011
,27017
and58000
Linux:
- x64 or arm64 architecture
- Docker or Podman installed
- CPU with AVX support
Mac:
- Not supported (yet)
2. Export your data from Discord
JSON exports are downloaded using DiscordChatExporter.
<details><summary><b>I am using GUI version of DiscordChatExporter</b></summary>-
Download the latest GUI version of DiscordChatExporter - most commonly used is
DiscordChatExporter.win-x64.zip
for 64-bit Windows -
If you want to export threads and forum posts, set
Show threads
option toAll
in the general settings of DiscordChatExporter
-
Obtain a Discord token by following this guide
-
Enter your Discord token to DiscordChatExporter and press Enter
-
If you have enabled
Show threads
option, load times are longer than expected, it will eventually load :) -
Select the channels you want to export (or press CTRL+A to select all channels)
-
Make sure that export format is set to
JSON
andFormat markdown
is disabled. EnableDownload assets
+Reuse assets
option to download images, videos and other types of assets.
- Download the latest CLI version of DiscordChatExporter - most commonly used is
DiscordChatExporter.Cli.win-x64.zip
for 64-bit Windows - Extract it.
- Open a terminal in the extracted folder by pressing ALT+D to highlight the path in the file explorer, then type type
cmd
and press Enter to open a terminal in that folder - Verify that DiscordChatExporter is working by running
DiscordChatExporter.Cli.exe --help
Export all accessible channels, threads and forum posts in a server:
DiscordChatExporter.Cli.exe exportguild --token DISCORD_TOKEN -g SERVER_ID --media --reuse-media --markdown false --format Json --include-threads All --output OUTPUT_FOLDER_PATH
Export private messages:
DiscordChatExporter.Cli.exe exportdm --token DISCORD_TOKEN --media --reuse-media --markdown false --format Json --output OUTPUT_FOLDER_PATH
TL;DR - always use
--media --reuse-media --markdown false --format Json
command line options for your exports to work with DCEF.
TIP: if you use Linux and want to automatically export your servers or DMs incrementally, you can try my slatinsky/DiscordChatExporter-incrementalBackup wrapper. You declare what you want to export in a config file and the wrapper will automatically export new messages from servers and DMs.
</details>3. Install DCEF
<details><summary><b>I am using Windows</b></summary>Only x64 architecture is supported. Won't work on Windows 7. Tested on Windows 10.
- Download the latest release from releases page.
- Extract the archive
- Move your exports downloaded in the first chapter to
exports/
folder. - Apply a registry tweak located in
registry_tweaks/change_260_character_path_limit_to_32767.reg
and restart your computer. This tweak will increase the path limit to 32767 characters.
BUG: images and other assets won't load if the path to the assets exceeds 260 characters even with the registry tweak applied. FIX - Place your DCEF folder to a shorter path.
BETA BUILDS: Want to try new features? Beta builds. Beta builds are automatically built from the latest commit on main
branch. (github account is needed to download them)
Both amd64
and arm64
architectures are supported. Also your CPU is required to support AVX instructions (some VPS providers don't support it). Works for Raspberry Pi 5, but not on Raspberry Pi 4b (does not support AVX).
If you use podman
instead of docker
, replace docker
with podman
in the commands.
- Pull the image from docker hub
docker pull slada/dcef:main
- Navigate to a folder with your exports downloaded in the first chapter
cd /path/to/your/exports
- Run the container. This command will mount your current working directory to exports folder inside the container
docker run --restart=always --volume "$(pwd):/dcef/exports" --volume dcef_cache:/dcef/cache --name dcef -p 127.0.0.1:21011:21011 -it slada/dcef:main
The app will autostart with your computer if you are using docker
. You need to create a systemd service to autostart the app if you use podman
.
Install python3 and nodejs. Make sure that your node --version
is at least v18.17.1
and your py --version
is at least Python 3.11.0
. Make sure Windows Terminal is installed (wt --version
). Windows terminal is installed by default on Windows 11.
Install nodemon
globally, which is used to automatically reload programs on change, by running npm install -g nodemon
. Verify that it is installed correctly by running nodemon --version
.
Then place your exports into src/exports/
folder. Create that folder if it doesn't exist.
Then run in cmd
the batch script src/RUN_DEV.bat
:
cd src
RUN_DEV.bat
This script will install Python dependencies in virtual environment (stored in src/_temp
folder) and will install frontend nodejs dependencies into src/dcef/frontend/node_modules/
. Then it will open windows terminal with split panes with all the services needed to develop the app.
- frontend - automatically reloads on change. If you add a new dependency into
src/frontend/package.json
, close the watcher, runnpm install
and rerunRUN_DEV.bat
- fastapi - automatically reloads on change. Delete
src/_temp/fastapi/venv/
folder if you change a dependencies. Then rerunRUN_DEV.bat
. - nginx - needs to be restarted each time a config is updated -
taskkill /f /im dcefnginx.exe
and rerunRUN_DEV.bat
again - mongodb - if you need to clear the database,
taskkill /f /im dcefmongod.exe
, deletesrc/_temp/mongodb/
folder and rerunRUN_DEV.bat
again (preprocess will recreate the database from the scratch) - preprocess - automatically reloads on change using nodemon. Delete
src/_temp/preprocess/venv/
folder if you change a dependencies. Then rerunRUN_DEV.bat
.
There are two ways to compile the Windows binaries:
- Fork the project and run the github actions (located in
.github/workflows/windows-build.yml
) yourself. Then download the built binaries from the Github website. - Or follow the steps in previous chapter about setting up development environment. Then run
BUILD_RELEASE.bat
script located insrc/
directory. This script will compile the binaries and place them intorelease/
folder. Test the binaries by runningrelease/dcef.exe
Also you can download build for each commit into main branch from my github actions. You need to be logged-in to Github account to download them
</details> <details><summary><b>[for developers] I want to build and run a docker image from the source code</b></summary>Prerequisites: You need docker and git installed.
Clone the source code:
git clone https://github.com/slatinsky/DiscordChatExporter-frontend
cd DiscordChatExporter-frontend
Pull new changes from the git repository:
git pull
docker rm dcef --force
docker build -t dcef .
Run it:
cd release/exports/
docker run --restart=always --volume "$(pwd):/dcef/exports" --volume dcef_cache:/dcef/cache --name dcef -p 127.0.0.1:21011:21011 -d dcef
cd ../..
TIP: In case you encounter any problems with outdated exports, you can try to rebuild the database by deleting dcef_cache
volume and then rerunning the container:
docker volume rm dcef_cache
</details>
4. Run DCEF
<details><summary><b>I am using Windows</b></summary>- Run
dcef.exe
- Wait for the program to process your exports. This will take several minutes
- After preprocessing is finished, a window with graphical interface will appear. The same interface will also be available in your browser at 127.0.0.1:21011
Open 127.0.0.1:21011 in your browser to see graphical interface. It may take a while for all your exports to be visible. Check docker logs dcef
to see the progress of the preprocessing.
TIP: change -p 127.0.0.1:21011:21011
to -p 21011:21011
in docker run
command to make the viewer available from other devices in your network.
5. Secure DCEF
<details><summary><b>I am using Linux</b></summary>The client-server architecture allows you to host the viewer on public servers. It is recommended to put the server behind another reverse proxy, such as nginx, that will handle SSL termination and authentication.
Restrict open ports using firewall:
Create firewall rules to open only TCP ports 22 (SSH) and 80 (HTTP) and enable firewall. If you want to use SSL, open port 443 (HTTPS) too.
ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable
Check if DCEF port is bound only to loopback interface (localhost 127.0.0.1
):
docker run ... -p 127.0.0.1:21011:21011 ...
Secure the viewer with basic authentication:
Create .htpasswd
file:
sudo apt install apache2-utils
htpasswd -c /etc/nginx/.htpasswd <username>
<password>
Change nginx site config to require authentication (add lines auth_basic
and auth_basic_user_file
):
server {
listen 80;
location / {
proxy_pass http://localhost:21011/;
auth_basic "Auth only";
auth_basic_user_file /etc/nginx/.htpasswd;
}
}
Restart nginx:
sudo systemctl restart nginx
NOTE: this is just a basic configuration and your password is sent in plaintext. If you own a domain, add SSL to encrypt the connection, use port 80 just to redirect to port 443. You can use free certificates from Let's Encrypt by configuring Certbot to automate the process. Find a more detailed guide on the internet.
</details>Upgrade DCEF to a new version
<details><summary><b>I am using Windows</b></summary>- Download the latest release zip from releases page - name of the zip should be
DiscordChatExporter-frontend-vX.Y.Z-win.zip
whereX.Y.Z
is the version number. - Delete everything (except
exports/
folder) in your DiscordChatExporter-frontend folder. - Move everything (except
exports/
folder) from the new release zip to your DiscordChatExporter-frontend folder.
Tip: some upgrades don't require the database to be rebuilt. If you want to speed up the upgrade process, don't delete _temp/mongodb/
folder. If the upgrade requires the database to be rebuilt, it will be handled automatically.
Pull the new image from the docker hub and restart the container.
cd path/to/your/exports/
docker rm dcef --force
docker image rm slada/dcef:main
docker pull slada/dcef:main
docker run --restart=always --volume "$(pwd):/dcef/exports" --volume dcef_cache:/dcef/cache --rm --name dcef -p 127.0.0.1:21011:21011 -it slada/dcef:main
</details>
Downgrade DCEF to an older version
<details><summary><b>I am using Windows</b></summary>- Download older release zip from releases page - name of the zip should be
DiscordChatExporter-frontend-vX.Y.Z-win.zip
whereX.Y.Z
is the version number. - Delete everything (except
exports/
folder) in your DiscordChatExporter-frontend folder. - Delete
_temp/
folder, which contains the database and other temporary files. This will force DCEF to rebuild the database from scratch and fill fix issues with the possibly incompatible database format. - Move everything (except
exports/
folder) from the release zip to your DiscordChatExporter-frontend folder.
Docker hub contains only the latest image built from the latest commit in main branch. if you want to use older version, you need to build the image from the source code.
Clone the git repository:
git clone https://github.com/slatinsky/DiscordChatExporter-frontend
cd DiscordChatExporter-frontend
Switch to the commit you want to build:
git checkout COMMIT_HASH
Remove the old image and the old database (dcef_cache
volume):
docker rm dcef --force
docker volume rm dcef_cache
Build the image:
docker build -t dcef .
Run the image:
cd path/to/your/exports/
docker run --restart=always --volume "$(pwd):/dcef/exports" --volume dcef_cache:/dcef/cache --name dcef -p 127.0.0.1:21011:21011 -d dcef
</details>
Uninstall DCEF
<details><summary><b>I am using Windows</b></summary>This is a portable app. Just delete the folder with the app. But keep your exports from the exports/
folder :).
Optionally also revert the registry tweak to decrease the path limit back to 260 characters. Run registry_tweaks/restore_260_character_path_limit.reg
and restart your computer. But this is really not necessary.
- Kill and delete the container
docker rm dcef --force
- Remove the volume
docker volume rm dcef_cache
- Remove the image
docker image rm slada/dcef:main
</details>
Troubleshooting
<details><summary><b>I am using Windows</b></summary>Check the logs See logs logs/dcef.log
.
Please check, if the top of the log contains line windows-runner: OK: All required ports are available.
. DCEF needs ports 21011
, 27017
, 58000
to be available. If you have any of these ports occupied, DCEF won't start or won't work properly.
Check if one of those ports is not excluded by running netsh int ipv4 show excludedportrange protocol=tcp
Find line found X json channel exports
- if this number is 0, you don't have any valid exports in /exports/
folder. Did you export in json
format instead of the default html
format? Did you apply the registry tweak and restarted your computer?
Find if there is line preprocess done
(at the end of lines processing <PATH_TO_JSON_FILE>
) - if you see this line, preprocessing step finished successfully. If you don't see this line, please check if there is any stacktrace in the logs. Stacktrace looks like this:
Traceback (most recent call last):
File "dcef/backend/preprocess/main_mongo.py", line 82, in <module>
main(input_dir, output_dir)
File "dcef/backend/preprocess/main_mongo.py", line 67, in main
raise Exception("Example stack trace exception")
Exception: Example stack trace exception
Please report this issue with the stacktrace attached.
The viewer is tested against standard install of Windows 10, so there may be issues if you have some non-standard Windows setup. [advanced users] If no other solution works for you, run Docker version of DCEF inside WSL2 and follow linux instructions.
</details> <details><summary><b>I am using Linux</b></summary>Check the logs
docker logs dcef
Find line found X json channel exports
- if this number is 0, you don't have any valid exports in /dcef/exports/
folder (inside the container). Did you mount the folder correctly? To debug this issue, get a shell inside the container and list the folder
docker exec -it dcef bash
cd /dcef/exports/
ls
Export should be visible in that /dcef/exports/
folder
FAQ
<details><summary><b>Some images and other assets won't display on Windows</b></summary>ISSUE: Files in /exports/
folder may exceed Windows path length limit of 260 characters. An issue with nginx server is preventing assets from being served correctly.
FIX: Move DCEF to a folder with shorter path, e.g. C:\Users\User\Downloads\DiscordChatExporter-frontend-vX.Y.Z-win\
-> C:\DCEF\
After you put your export to exports/
folder, don't remove and don't move them. DCEF keeps track of files and other assets in this folder and if you remove or move them, they won't show up correctly in DCEF, because the old paths would become invalid.
TL;DR - only adding new files to exports/
folder is supported. If you want to remove or move files, you need to clear the database afterwards.
Clearing database on Windows:
- close DCEF
- delete
_temp/mongodb/
folder - start DCEF again
Clearing database on Linux:
- kill DCEF container
- remove
dcef_cache
volume (docker volume rm dcef_cache
) - start DCEF container again
DCEF is not just an simple viewer. It needs to preprocess the data into a database, so you can search through the messages. This process can take a long time for large exports. The next time you run DCEF, the startup will be fast (it will only process new exports).
Impatient? Navigate to http://127.0.0.1:21011/
in your browser to see already processed exports :)
Mac is not supported, but this pull request may help you
</details> <details><summary><b>DCEF is detected as a malware</b></summary>DCEF is not a malware. It's a false positive. The project is open source, you can check the source code yourself.
The windows release is exactly the same as the ones built on Github's servers by github actions. Executables (nginx.exe
, mongod.exe
) are sourced from their official websites. I upload releases manually, but the zip is exactly the same as the last successful build on github actions.
Docker image is built on Github's servers github actions github actions too. This action directly uploads the image to docker hub.
Tinfoil hat on? Replace nginx.exe
, mongod.exe
, msvcp140.dll
and vcruntime140_1.dll
with your own trusted copy. Then compile your own version from source code.
Thanks
- Tyrrrz/DiscordChatExporter - for a great export tool
And for other great technologies used in this project - SvelteKit, Node.js, PyInstaller, nginx, MongoDB, Docker
Related projects
- Tyrrrz/DiscordChatExporter - exporter for your Discord chats
- Roachbones/discordless - real time man-in-the-middle exporter
- mlomb/chat-analytics - analytics for your Discord chats
- slatinsky/DiscordChatExporter-incrementalBackup - incrementally export your Discord chats for DiscordChatExporter-frontend
License
GNU GENERAL PUBLIC LICENSE. See LICENSE for more details.
This product contains software provided by NGINX and its contributors.
DiscordChatExporter-frontend is not affiliated with Discord. Discord is a registered trademark of Discord Inc.
Contributing
Feel free to open issues and pull requests.
<details><summary><b>Short guide, how to contribute</b></summary>- Fork the repository
- Create a new branch
- Implement your changes
- Commit and push your changes
- Create a pull request
If you find this project useful, give it a star ⭐. Thank you!