I firstly logged out of the games and then began a search of my computer.
I used Windows Explorer to search for the proper folder. I selected my C drive and searched for the downloaded puzzles, I found them buried in the users folders AppData/Local/Packages/Microsoft.MicrosoftJigsaw_*******/LocalState/DownloadedPuzzels - I then simply deleted them from this folder.
Once I had deleted them I went back into the game and the puzzle I had deleted was no longer showing in my collections, I was able to download the puzzle again.
I hope this helps