get-zones/preview not seeing all Cloudflare records

[welbow]

I'm trying to migrate a test zone to dnscontrol with DNS hosted at Clouflare. The API token appears properly configured (more on that below). However, get-zones is not showing all records from Cloudflare DNS. I tried exporting from CF into a BIND-style file and converted that to DJS. dnscontrol preview is trying to create the "missing" records.

Things I've tried:

• --debug - just kept printing the dnscontrol version no matter which command I ran
• export CLOUDFLAREAPI_DEBUG="1" - this shows me that dnscontrol can see the records (missing records are mx records, DKIM txt record, NS and SOA records), but they don't ever show up in get-zones. I have no IGNORE* statements in my dnsconfig.js
• Tried the docker version in case my local install is wonky - same outcome.

Any ideas?

[welbow]

I have an update. I think I was running into issue #3850 - but didn't realize that was expected behavior and was concerned if I didn't have those records in my JS, dnscontrol might delete them. Commenting them out fixed the preview - however, there is one CF SPF TXT record that wasn't marked readonly so I had to keep that in my JS.

I figured this out by noting the readonly:true in the cloudflare api debug and looking at the code. I didn't notice anything about this on the Cloudflare page in the dnscontrol documentation. Could this be an enhancement for documentation?

Regarding --debug, I still can't get that to do anything useful.

root@host:~/dns-example# dnscontrol --debug preview
dnscontrol version DNSControl version v4.36.1
root@host:~/dns-example# dnscontrol preview --debug
Incorrect Usage: flag provided but not defined: -debug

root@host:~/dns-example#

[tlimoncelli]

@welbow Reguarding the --debug flag:

First, the --debug flag (and all global flag must be before the subcommand. That is, dnscontrol --debug preview not dnscontrol preview --debug.

As far as -v, I see the problem. -v is a short flag for 2 different things: --debug and --version.

GLOBAL OPTIONS:
   --debug, -v        Enable debug logging
   --allow-fetch      Enable JS fetch(), dangerous on untrusted code!
   --disableordering  Disables update reordering
   --no-colors        Disable colors
   --help, -h         show help
   --version, -v      print the version

Our flag parser will always pick -v as shorthand for --version.

The solution is to use --debug and not the short cut. I'm going to remove -v as a shortcut for --debug since that aligns with the way the flag parser (github.com/urfave/cli) works.

[tlimoncelli]

The other point I'd like to make about the --debug flag is that it doesn't do very much. That said, let's look at the specific problem you've raised:

export CLOUDFLAREAPI_DEBUG="1" - this shows me that dnscontrol can see the records (missing records are mx records, DKIM txt record, NS and SOA records), but they don't ever show up in get-zones. I have no IGNORE* statements in my dnsconfig.js

My suggestion is to manually update dnsconfig.js until "dnscontrol preview" runs clean.

• You can probably skip SOA records; Cloudflare will generate them for you if you don't include them in dnsconfig.js.
• You should be able to skip NS records at the apex (@) since the Cloudflare provider automatically gathers that info.

I'd be interested in knowing which records you end up having to add manually. (and, yes, this should be better documented. Would you be interested in proposing a draft?).

Tom

CC @cafferata

[welbow]

@tlimoncelli, thanks much for your reply. I was using --debug before the subcommand per the -h help page that specified it as a global command. However, doing that gave me odd output as I mentioned above:

root@host:~/dns-example# dnscontrol --debug preview
dnscontrol version DNSControl version v4.36.1

It looked like the command-line parsing was doing something odd.

RE our domain, I found out that anything that CF marks readonly should not be in our JS config. For this domain specifically, that is the 3 MX records and the CF DKIM record. This domain is using mail routing via CF. However, the CF SPF record was required in our config to get a clean dnscontrol preview.

        // Cloudflare does not mark this record as readonly, so we have to manage it.
        , TXT("@", "v=spf1 include:_spf.mx.cloudflare.net ~all",TTL(1))

For documentation, a note close to the top of the Cloudflare provider page would be helpful. Some key points (of course, fix anything I misunderstand):

• Cloudflare marks some DNS entries as readonly - this was visible only with the debug envvar set
• dnscontrol sees that, and purposefully excludes them from consideration (get-zone/preview/push)
• Examples of readonly records are SOA, NS, MX/DKIM records (when Cloudflare mail routing is used), etc.

Also, consider noting the envvar on the provider page as well. I only found that by finding another github issue that suggested using it. Until I saw all DNS records in the output from the CF API calls, I was concerned that my API token was somehow misconfigured/not working.

[tlimoncelli]

Yes, the --debug feature is inaccessible right now. Try an older version or wait until the next release. Sadly I can't figure out a workaround with the current release. (On the bright side (??)... the --debug flag does very little to help in this situation.)

Thanks for the doc suggestion! Please take a look at #4146 and critque!

[welbow]

Thanks @tlimoncelli - the doc updates look good and glad to know a debug fix is in the works :-D