Skip to contents

NARWC data often records a value once and leaves it blank until it changes — LEGTYPE is entered as 2 at the start of a census line and the rows beneath it are empty until the leg type changes. This fills those blanks, within a survey day and file, so that downstream code sees the state each record was actually flown under.

Usage

fill_narwc(
  dat,
  columns = narwc_fill_columns(),
  by = NULL,
  direction = c("downup", "down", "up"),
  quiet = FALSE
)

Arguments

dat

A data frame of NARWC survey data, in survey order.

columns

Columns to fill. Defaults to narwc_fill_columns(). Anything in narwc_never_fill() is an error rather than a silent omission. LEGSTAGE is accepted and handed to fill_legstage().

by

Grouping columns. NULL (default) uses FILEID and DATE where present.

direction

"downup" (default), "down", or "up", as tidyr::fill().

quiet

Suppress the report of what was filled. Default FALSE.

Value

dat with the requested columns filled, plus LEGSTAGE_FILLED when LEGSTAGE was requested.

Grouping is the whole point

An ungrouped fill runs the length of the file. The sea state from the last record of one survey day carries into the first records of the next; a leg number carries across a FILEID boundary into a different survey entirely. Neither is recoverable afterwards, because the filled value is indistinguishable from a recorded one.

The scripts this package was rewritten from filled these columns with no grouping at all (DataExploration.R:52, :71). by defaults to FILEID and DATE, and if neither is present this function warns rather than quietly filling across everything.

Direction, and what "up" actually does

tidyr::fill() semantics. "down" is the direction the recording convention justifies: a blank means "as above".

"downup" fills down first and then fills up, so the only values it fills backwards are those before the first recorded value in a group. That is a smaller claim than it sounds, but it is still a guess: the state before anything was logged is genuinely unknown, and back-filling asserts it matched whatever was recorded first. Where a day's records begin on transit before the first LEGTYPE is entered, back-filling a 2 would mark that transit as census effort.

For that reason the leg columns LEGTYPE and LEGNO are never filled backwards, whatever direction says. Records before the first leg code of a group are transit out to the survey area by convention, and make_leg_id() leaves them outside any line. The backward pass applies to the sighting-condition, platform and design columns only.

Because those are the inferred values rather than the recovered ones, the report counts them separately. A large backward count is worth looking at.

LEGSTAGE goes through fill_legstage()

LEGSTAGE is not in the default set. Carrying its value forward marks every record of a line "begin line" when only the first record carries a 1, and since on-effort eligibility is LEGSTAGE == 2 the whole line would drop out of every distance calculation. When LEGSTAGE is named in columns it is not filled as a value: after the other columns are filled, the data are passed to fill_legstage(), which writes 2 only where a line is continuing and adds LEGSTAGE_FILLED. direction does not apply to it.

References

Kenney, R.D. (2023) The North Atlantic Right Whale Consortium Database: A Guide for Users and Contributors, Version 8. NARWC Reference Document 2023-01.

Examples

dat <- data.frame(
  FILEID = "A", DATE = as.Date("2024-04-01"), EVENTNO = 1:5,
  LEGTYPE = c(2, NA, NA, 1, NA), BEAUFORT = c(3, NA, NA, NA, NA)
)
fill_narwc(dat)
#> `fill_narwc()` filled 7 values, grouped by FILEID, DATE.
#>   carried forward:  7
#>   BEAUFORT 4, LEGTYPE 3
#>   FILEID       DATE EVENTNO LEGTYPE BEAUFORT
#> 1      A 2024-04-01       1       2        3
#> 2      A 2024-04-01       2       2        3
#> 3      A 2024-04-01       3       2        3
#> 4      A 2024-04-01       4       1        3
#> 5      A 2024-04-01       5       1        3

# Refuses to replicate a sighting
try(fill_narwc(dat, columns = "NUMBER"))
#> Error in fill_narwc(dat, columns = "NUMBER") : 
#>   These columns must not be carried forward: `NUMBER`.
#> They are per-record measurements, not survey state. Filling a sighting column replicates one detection onto every row beneath it; filling a position or time fabricates a measurement. See `?narwc_fill_columns`.