Creating what-if stories requires reasoning about prior statements and
possible outcomes of the changed conditions. One can easily generate coherent
endings under new conditions, but it would be challenging for current systems
to do it with minimal changes to the original story. Therefore, one major
challenge is the trade-off between generating a logical story and rewriting
with minimal-edits. In this paper, we propose EDUCAT, an editing-based
unsupervised approach for counterfactual story rewriting. EDUCAT includes a
target position detection strategy based on estimating causal effects of the
what-if conditions, which keeps the causal invariant parts of the story. EDUCAT
then generates the stories under fluency, coherence and minimal-edits
constraints. We also propose a new metric to alleviate the shortcomings of
current automatic metrics and better evaluate the trade-off. We evaluate EDUCAT
on a public counterfactual story rewriting benchmark. Experiments show that
EDUCAT achieves the best trade-off over unsupervised SOTA methods according to
both automatic and human evaluation. The resources of EDUCAT are available at:
https://github.com/jiangjiechen/EDUCAT.