URL Validation

Overview

URL validation lets SEARCH v4 confirm that the URL it receives is exactly the URL you created. You add a validation parameter, calculated from the URL and your private key with SHA-256. SEARCH v4 recalculates the value internally. If it matches, the request is processed. If it does not, the request is rejected. Nothing needs to be integrated on the receiving side. You only need to generate the validation value and call the resulting URL from your client-side script.

What it protects against

Validation is designed to catch changes made to the URL after you create it and before SEARCH v4 receives it, for example by a redirect or another intermediary in the path. If the URL is altered in any way, the validation no longer matches and the request is rejected. It relies on your private key staying secret. Anyone who has the key can create a valid validation value, so see Protecting your private key.

URL validation is available exclusively for SEARCH v4, and only for clients that have it enabled. When it is enabled, a request without the validation parameter fails with the message Required parameter(s) : validation should be provided.

How the validation value is calculated

Build the full URL

Use the complete URL, including the scheme, host, path, and every query parameter, in the exact order you will send them. Do not include validation yet.

Append your private key

Add your private key directly to the end of the URL, with no separator.

Calculate SHA-256

Hash the combined string as UTF-8. The result is a 64-character lowercase hexadecimal string.

Add it as the last parameter

Append &validation= followed by the hash to the original URL. Call that URL exactly as it is.

validation = SHA-256( URL + privateKey )
signedUrl  = URL + "&validation=" + validation

Any change to the URL after it is signed, such as reordering parameters, adding a parameter, or changing the host, makes the validation fail. Sign the final URL and call it unchanged.

Manual example

You can try the process by hand, for example to test your integration.

URL="https://prod.rtymgt.com/api/v4/respondents/search/YOUR_PUBLISHABLE_KEY?sy_nr=YOUR_SURVEY_NUMBER&sn_ud=YOUR_SESSION_ID&dupe_details=1&df_details=1"
PRIVATE_KEY="YOUR_PRIVATE_KEY"

# SHA-256 of the URL followed directly by the private key (no separator).
# On macOS, use: shasum -a 256
VALIDATION=$(printf '%s%s' "$URL" "$PRIVATE_KEY" | sha256sum | cut -d' ' -f1)

# Call the signed URL
curl "$URL&validation=$VALIDATION"

Without the validation parameter, the same request returns the error message above. With it, the request returns the normal search result.

Protecting your private key

Your private key is a secret. Anyone who has it can create a valid validation value for any URL. This integration runs in the respondent’s browser, and anything in client-side code can be read. The sample below keeps the key in a constant for demonstration only. How your integration provides the private key to the script without exposing it is up to you, so decide this before you go live.

Do not treat the sample’s PRIVATE_KEY constant as a way to protect the key. If the key can be read from your page, it can be used to sign changed URLs.

How it works

Prerequisites

A Research Defender publishable key, used for the library and in the URL path
A Research Defender private key, used to calculate validation
A client account with URL validation enabled for SEARCH v4
Your survey number (sy_nr) and the respondent’s session ID (sn_ud)
A page served over HTTPS, or http://localhost for testing. The browser’s SHA-256 function is only available in a secure context, so opening the file directly from disk (file://) will not work.

Request

string

GET https://{environment}.rtymgt.com/api/v4/respondents/search/{publishable_key}

string

required

The SHA-256 validation value, as 64 lowercase hexadecimal characters. It must be the last parameter.

string

required

The survey number.

string

required

The respondent’s session ID.

string

Set to "1" to enable.

string

Set to "1" to enable.

string

required

The string returned by pass_params(). Send it as the name of a query parameter with an empty value.

Every parameter you send is part of the signed URL. If you add another parameter, for example encrypt=1 for an encrypted response, add it in buildSearchUrl() so it is included before the URL is signed.

Browser integration

// ---------------------------------------------------------------
// Configuration
// ---------------------------------------------------------------
const ENVIRONMENT = "prod"; // "staging" or "prod"
const PUBLISHABLE_KEY = "YOUR_PUBLISHABLE_KEY";
const LIBRARY_VERSION = "2.0.1";

// Your PRIVATE key. This sample keeps it in a constant for demonstration only.
// Anything in client-side code can be read, so decide how your integration
// provides this value to the script without exposing it.
const PRIVATE_KEY = "YOUR_PRIVATE_KEY";

// Replace with the values for the respondent you are checking.
const SURVEY_NUMBER = "YOUR_SURVEY_NUMBER"; // sy_nr
const SESSION_ID = "YOUR_SESSION_ID";       // sn_ud

// Time to let the library collect its datasets before reading them.
const DATASET_READY_DELAY_MS = 1000;

const BASE_URL = `https://${ENVIRONMENT}.rtymgt.com`;
const LIBRARY_URL = `${BASE_URL}/static/js/rd_sys_util.min.js?v=${LIBRARY_VERSION}`;
const SEARCH_ENDPOINT = `${BASE_URL}/api/v4/respondents/search/${PUBLISHABLE_KEY}`;

// ---------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

/**
 * Parses a response body. If the body has a stray trailing character
 * that breaks JSON.parse, it is trimmed and parsing is retried.
 */
function parseResponse(text) {
  try {
    return JSON.parse(text);
  } catch {
    return JSON.parse(text.slice(0, -1));
  }
}

/** Percent-encodes a query string name or value (RFC 3986). */
function encode(value) {
  return encodeURIComponent(value).replace(
    /[!'()*]/g,
    (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`
  );
}

/** Returns the SHA-256 hash of a string as 64 lowercase hex characters. */
async function sha256Hex(text) {
  const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(text));
  return Array.from(new Uint8Array(digest), (b) => b.toString(16).padStart(2, "0")).join("");
}

/** Loads the library, starts data collection, and returns its pass-through key. */
async function collectLibraryData() {
  const { init, pass_params } = await import(LIBRARY_URL);

  if (typeof init !== "function" || typeof pass_params !== "function") {
    throw new Error("Research Defender library did not load correctly.");
  }

  await init({ key: PUBLISHABLE_KEY, env: ENVIRONMENT });

  // Give the library time to finish collecting its datasets.
  await sleep(DATASET_READY_DELAY_MS);

  const pass = pass_params();
  console.log("[rd] pass_params computed:", pass ? `${pass.length} chars` : "empty");
  return pass;
}

/** Builds the SEARCH v4 URL. The parameter order is part of what is signed. */
function buildSearchUrl({ surveyNumber, sessionId, pass }) {
  const params = [
    ["sy_nr", surveyNumber],
    ["sn_ud", sessionId],
    ["dupe_details", "1"],
    ["df_details", "1"],
  ];

  // The value returned by pass_params() is sent as a parameter name with an
  // empty value.
  if (pass) {
    params.push([pass, ""]);
  }

  const query = params.map(([name, value]) => `${encode(name)}=${encode(value)}`).join("&");
  return `${SEARCH_ENDPOINT}?${query}`;
}

/** Appends validation = SHA-256 hex of (URL + private key) as the last parameter. */
async function signUrl(url) {
  const validation = await sha256Hex(url + PRIVATE_KEY);
  return `${url}&validation=${validation}`;
}

/** Calls the signed URL exactly as built. Do not modify it after signing. */
async function search(signedUrl) {
  const response = await fetch(signedUrl, {
    method: "GET",
    credentials: "include", // send Research Defender cookies
  });

  if (!response.ok) {
    throw new Error(`Search request failed with status ${response.status}`);
  }

  return parseResponse(await response.text());
}

// ---------------------------------------------------------------
// Usage
// ---------------------------------------------------------------
const pass = await collectLibraryData();

const url = buildSearchUrl({
  surveyNumber: SURVEY_NUMBER,
  sessionId: SESSION_ID,
  pass,
});
const signedUrl = await signUrl(url);

const result = await search(signedUrl);

console.log("Threat Potential:", result.Respondent?.threat_potential);
console.log("Duplicate Flag:", result.Surveys?.[0]?.flag);
console.log("Survey Number:", result.Surveys?.[0]?.survey_number);

collectLibraryData() is the same function used in the advanced integration.
buildSearchUrl() builds the URL with the parameters in a fixed order and percent-encodes each name and value.
signUrl() calculates validation with the browser’s built-in crypto.subtle and appends it as the last parameter.
search() calls the signed URL exactly as built, with credentials included so Research Defender cookies are sent.

Full HTML page

URL validation integration (HTML)

validated-client.html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title>Research Defender – URL Validation Integration</title>
</head>
<body>
  <h1>Research Defender – URL Validation Integration</h1>
  <pre id="output">Loading…</pre>

  <script type="module">
    // ---------------------------------------------------------------
    // Configuration
    // ---------------------------------------------------------------
    const ENVIRONMENT = "prod"; // "staging" or "prod"
    const PUBLISHABLE_KEY = "YOUR_PUBLISHABLE_KEY";
    const LIBRARY_VERSION = "2.0.1";

    // Your PRIVATE key. This sample keeps it in a constant for demonstration only.
    // Anything in client-side code can be read, so decide how your integration
    // provides this value to the script without exposing it.
    const PRIVATE_KEY = "YOUR_PRIVATE_KEY";

    // Replace with the values for the respondent you are checking.
    const SURVEY_NUMBER = "YOUR_SURVEY_NUMBER"; // sy_nr
    const SESSION_ID = "YOUR_SESSION_ID";       // sn_ud

    // Time to let the library collect its datasets before reading them.
    const DATASET_READY_DELAY_MS = 1000;

    const BASE_URL = `https://${ENVIRONMENT}.rtymgt.com`;
    const LIBRARY_URL = `${BASE_URL}/static/js/rd_sys_util.min.js?v=${LIBRARY_VERSION}`;
    const SEARCH_ENDPOINT = `${BASE_URL}/api/v4/respondents/search/${PUBLISHABLE_KEY}`;

    // ---------------------------------------------------------------
    // Helpers
    // ---------------------------------------------------------------
    const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

    /**
     * Parses a response body. If the body has a stray trailing character
     * that breaks JSON.parse, it is trimmed and parsing is retried.
     */
    function parseResponse(text) {
      try {
        return JSON.parse(text);
      } catch {
        return JSON.parse(text.slice(0, -1));
      }
    }

    /** Percent-encodes a query string name or value (RFC 3986). */
    function encode(value) {
      return encodeURIComponent(value).replace(
        /[!'()*]/g,
        (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`
      );
    }

    /** Returns the SHA-256 hash of a string as 64 lowercase hex characters. */
    async function sha256Hex(text) {
      const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(text));
      return Array.from(new Uint8Array(digest), (b) => b.toString(16).padStart(2, "0")).join("");
    }

    /** Loads the library, starts data collection, and returns its pass-through key. */
    async function collectLibraryData() {
      const { init, pass_params } = await import(LIBRARY_URL);

      if (typeof init !== "function" || typeof pass_params !== "function") {
        throw new Error("Research Defender library did not load correctly.");
      }

      await init({ key: PUBLISHABLE_KEY, env: ENVIRONMENT });

      // Give the library time to finish collecting its datasets.
      await sleep(DATASET_READY_DELAY_MS);

      const pass = pass_params();
      console.log("[rd] pass_params computed:", pass ? `${pass.length} chars` : "empty");
      return pass;
    }

    /** Builds the SEARCH v4 URL. The parameter order is part of what is signed. */
    function buildSearchUrl({ surveyNumber, sessionId, pass }) {
      const params = [
        ["sy_nr", surveyNumber],
        ["sn_ud", sessionId],
        ["dupe_details", "1"],
        ["df_details", "1"],
      ];

      // The value returned by pass_params() is sent as a parameter name with an
      // empty value.
      if (pass) {
        params.push([pass, ""]);
      }

      const query = params.map(([name, value]) => `${encode(name)}=${encode(value)}`).join("&");
      return `${SEARCH_ENDPOINT}?${query}`;
    }

    /** Appends validation = SHA-256 hex of (URL + private key) as the last parameter. */
    async function signUrl(url) {
      const validation = await sha256Hex(url + PRIVATE_KEY);
      return `${url}&validation=${validation}`;
    }

    /** Calls the signed URL exactly as built. Do not modify it after signing. */
    async function search(signedUrl) {
      const response = await fetch(signedUrl, {
        method: "GET",
        credentials: "include", // send Research Defender cookies
      });

      if (!response.ok) {
        throw new Error(`Search request failed with status ${response.status}`);
      }

      return parseResponse(await response.text());
    }

    // ---------------------------------------------------------------
    // Run
    // ---------------------------------------------------------------
    const output = document.getElementById("output");

    try {
      const pass = await collectLibraryData();

      const url = buildSearchUrl({
        surveyNumber: SURVEY_NUMBER,
        sessionId: SESSION_ID,
        pass,
      });
      const signedUrl = await signUrl(url);

      const result = await search(signedUrl);

      console.log("Full results:", result);

      const survey = result.Surveys?.[0];
      output.textContent = [
        `Threat Potential: ${result.Respondent?.threat_potential}`,
        `Duplicate Flag:   ${survey?.flag}`,
        `Survey Number:    ${survey?.survey_number}`,
        "",
        "Full response:",
        JSON.stringify(result, null, 2),
      ].join("\n");
    } catch (error) {
      console.error(error);
      output.textContent = `Error: ${error.message}`;
    }
  </script>
</body>
</html>

Replace the placeholder values at the top of the script, then serve the file over HTTPS or http://localhost.

Security recommendations

Treat the private key as a secret. Decide how your script receives it without exposing it, and never commit it to source control.
Do not log the private key or the string you hash.
Sign the final URL. Add every parameter before signing, and call the signed URL unchanged.

Troubleshooting

Symptom	Likely cause
`Required parameter(s) : validation should be provided.`	The request has no `validation` parameter, and validation is enabled for your account.
Validation is rejected	The URL changed after signing, the private key is wrong, or the signed string differs from the URL that was sent.
Rejected only after the URL passes through a redirect or intermediary	The redirect or intermediary is modifying, reordering, re-encoding, or dropping parameters. Compare the URL SEARCH v4 receives with the signed URL.
Works in a manual test but not in your integration	Check the parameter order, the percent-encoding of the `pass_params()` name, and that the same host (`staging` or `prod`) is used for signing and calling.
`crypto.subtle` is undefined	The page is not served over HTTPS or `http://localhost`.

SEARCH

REVIEW

ACTIVITY

PREDUPE

ACTIVATION

Getting Started

Overview

What it protects against

How the validation value is calculated

Manual example

Protecting your private key

How it works

Prerequisites

Request

Browser integration

Full HTML page

Security recommendations

Troubleshooting

​Overview

​What it protects against

​How the validation value is calculated

​Manual example

​Protecting your private key

​How it works

​Prerequisites

​Request

​Browser integration

​Full HTML page

​Security recommendations

​Troubleshooting

Overview

What it protects against

How the validation value is calculated

Manual example

Protecting your private key

How it works

Prerequisites

Request

Browser integration

Full HTML page

Security recommendations

Troubleshooting