Your readers, and you, access the blog content using the URL of the blog, and require use of the DNS system to do that.
However, Google doesn't host the DNS, and DNS hosting is an essential part of the custom domain setup process. Lack of proper DNS setup, for your custom domain, may cause any of several results - none of them good.
Custom domain publishing sets up 3 links, within the Google domain database.
When you publish your blog to a custom domain, you're setting up 3 links within Blogger / Google.
- All internal blog links, such as the blog feed URLs, and all links from any part of the blog to the others, are converted to point to the custom domain URL.
- A "301 Moved Permanently" redirect is setup for the original BlogSpot URL, pointing to the custom domain URL.
- An entry in "ghs.google.com" is added, pointing the custom domain URL to the blog.
All 3 of those links are purely within Google. For any, or all, of those links to work, you have to have a DNS entry for the custom domain URL, defined outside Google, pointing to Google.
With the domain successfully published, you can setup a secondary URL.
When you get the blog successfully published, you can optionally define a second URL to address the blog. If you published the blog to "www.mydomain.com", you can select "Redirect mydomain.com to www.mydomain.com.", and "mydomain.com" becomes an secondary URL. Here too, the Redirect is purely within Blogger.
You will have to have a DNS entry for the redirected URL, defined outside Google, pointing to Google.
If you're going to select "Redirect www.mydomain.com to mydomain.com.", you have to have properly setup DNS entries for both "mydomain.com" and "www.mydomain.com", and these entries must point into Google. If you don't have both URLs defined, or if both entries don't point into Google, you can expect to see a dear old friend
Another blog is already hosted at this address.
If you get the latter error, you either won't be able to publish the blog to the custom domain, or you won't be able to redirect the secondary alias. And that will possibly bring you to a second dear old friend
Not Foundor possibly
The errors discussed can have multiple causes - including incorrect DNS.
Both errors - "Another blog is already hosted at this address." and "Not Found Error 404" / "Forbidden Error 403" - can have multiple causes. If you see either error, it may not always cause / be caused by the other, and neither will be exclusively caused by bad DNS. But if you don't get your DNS setup properly, you can count on seeing one or both.
When you see "Another blog is already hosted at this address.", and it's caused by bad DNS, you will have to diagnose and fix the DNS problem, before you can reset the problem in Blogger. Whether you're using the Custom Domain Reset form, or the Google Apps domain setting recyle procedure, you have to get the DNS setup properly, first.
Don't waste your time, get your DNS working. DNS hosting is an essential part of the custom domain setup process.