RootsMagic 9 has the same file extension (.rmtree) as RootsMagic 8, but a different file format. When the RootsMagic 8 database is opened it will let you know that it needs to convert the file to the new format. You will also be advised to make a backup in the previous version before converting. If you continue RootsMagic will convert the file and open it. If you cancel, RootsMagic will not convert the file (it will leave it in the older format and won't open the file).
Note: Once the RootsMagic 8 database is converted to the new RootsMagic 9 file format it will no longer open in the previous version.
The RootsMagic 8 backup (.rmtree) needs to be restored inside of RootsMagic 9 by going to Files, Restore from backup. During the restore process it will create a new (.rmtree) file, convert the file, and leave the RootsMagic 8 backup file intact.