Baskets, alongside which exile once began, cannot be relocated or removed. They are a part of Grim Soul's story and serve as a symbol of the beginning of the Exile's journey in the Plaguelands. Don't forget that baskets may also be used for their intended purpose to store the loot you've obtained.