Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ PRIVATE_ORG=
# Leave unset for github.com. For GHE.com Data Residency, set GITHUB_SERVER_URL
# to your tenant URL (e.g. https://acme.ghe.com). For GHES, set it to your
# server URL (e.g. https://ghes.example.com). GITHUB_API_URL is derived
# automatically but can be overridden if needed.
# automatically but can be overridden if needed. GraphQL is derived from the
# REST API base and uses /api/graphql on GHES.
# NEXT_PUBLIC_* variants must also be set at build time (Docker build args) so
# client bundles and UI links target the correct host.
GITHUB_SERVER_URL=
Expand All @@ -42,7 +43,9 @@ NEXT_PUBLIC_GITHUB_API_URL=

# Committer email domain used on sync commits. Defaults to
# `users.noreply.github.com`. Set explicitly for GHE/GHES (the exact value
# depends on instance configuration).
# depends on instance configuration). If you leave it unset on a non-github.com
# deployment, the app logs a warning and still falls back to the github.com
# noreply domain for compatibility.
GITHUB_USER_EMAIL_DOMAIN=

# Used to skip branch protection creation if organization level branch protections are used instead
Expand Down
5 changes: 5 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@ WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

ARG NEXT_PUBLIC_GITHUB_SERVER_URL
ARG NEXT_PUBLIC_GITHUB_API_URL

ENV NEXT_TELEMETRY_DISABLED=1
ENV NEXT_PUBLIC_GITHUB_SERVER_URL=$NEXT_PUBLIC_GITHUB_SERVER_URL
ENV NEXT_PUBLIC_GITHUB_API_URL=$NEXT_PUBLIC_GITHUB_API_URL

RUN npm run build
RUN npm prune --omit=dev
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,11 @@ GITHUB_SERVER_URL=https://acme.ghe.com
# Required for client-side hooks and UI links to point at the correct host.
NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com

# Optional. Auto-derived from GITHUB_SERVER_URL:
# Optional REST API base URL. Auto-derived from GITHUB_SERVER_URL:
# github.com -> https://api.github.com
# <tenant>.ghe.com -> https://api.<tenant>.ghe.com
# <ghes-host> -> https://<ghes-host>/api/v3
# GraphQL is derived from this value and uses /api/graphql on GHES.
# Override only if the auto-derivation does not match your instance.
GITHUB_API_URL=
NEXT_PUBLIC_GITHUB_API_URL=
Expand All @@ -131,7 +132,8 @@ GITHUB_USER_EMAIL_DOMAIN=users.noreply.acme.ghe.com
Notes:

- The OAuth App / GitHub App, organizations, members and forks must all live on the same GHE instance.
- The `NEXT_PUBLIC_*` variables are inlined into the client bundle at build time. When building the Docker image, pass them as build args (e.g. `--build-arg NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com`) and update the `Dockerfile` to forward them into the `npm run build` step.
- The `NEXT_PUBLIC_*` variables are inlined into the client bundle at build time. When building the Docker image, pass them as build args (e.g. `--build-arg NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com`). The bundled `Dockerfile` already forwards them into the `npm run build` step.
- If you leave `GITHUB_USER_EMAIL_DOMAIN` unset on a non-github.com deployment, the app still falls back to `users.noreply.github.com` for compatibility, but it now logs a warning so you can correct the configuration.
- The local webhook relay (`npm run webhook`) uses `github-app-webhook-relay-polling` against the GitHub App hook deliveries endpoint. It is best-effort on GHE; in production, use real webhook deliveries configured directly on your GitHub App.

## Usage
Expand Down
2 changes: 1 addition & 1 deletion docs/developing.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ This will create an optimized production build of the app in the `out` directory

### Building for GHE.com / GHES

The `NEXT_PUBLIC_GITHUB_SERVER_URL` and `NEXT_PUBLIC_GITHUB_API_URL` env vars are inlined into the client bundle at build time. When targeting a GHE.com Data Residency tenant or a GHES instance, you must set them before running `npm run build` (or pass them as Docker build args). For example:
The `NEXT_PUBLIC_GITHUB_SERVER_URL` and `NEXT_PUBLIC_GITHUB_API_URL` env vars are inlined into the client bundle at build time. When targeting a GHE.com Data Residency tenant or a GHES instance, you must set them before running `npm run build` (or pass them as Docker build args). The bundled `Dockerfile` already forwards these build args into `npm run build`. For example:

```sh
NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com \
Expand Down
3 changes: 3 additions & 0 deletions scripts/webhook-relay.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ if (!process.env.PUBLIC_ORG) {

const url = `${process.env.NEXTAUTH_URL}/api/webhooks`

// Keep this fallback in sync with deriveApiUrlFromServerUrl in
// src/utils/github-urls.ts. The relay only needs the REST API base URL; GraphQL
// callers must use /api/graphql on GHES.
const deriveApiUrl = (serverUrl) => {
try {
const u = new URL(serverUrl)
Expand Down
68 changes: 38 additions & 30 deletions src/app/api/auth/lib/nextauth-options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,43 @@ export const refreshAccessToken = async (

const apiBaseUrl = getGitHubApiUrl()

export const createGitHubUserinfoRequest =
(apiBaseUrl: string) =>
async ({
client,
tokens,
}: {
client: { userinfo: (accessToken: string) => Promise<unknown> }
tokens: { access_token?: string | null }
}) => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const profile = (await client.userinfo(tokens.access_token!)) as any

if (!profile.email) {
try {
const res = await fetch(`${apiBaseUrl}/user/emails`, {
headers: {
Authorization: `token ${tokens.access_token}`,
'User-Agent': 'private-mirrors-app',
},
})

if (res.ok) {
const emails: Array<{
email: string
primary: boolean
verified: boolean
}> = await res.json()
profile.email = (emails.find((e) => e.primary) ?? emails[0])?.email
}
} catch (error) {
authLogger.warn('Failed to fetch user emails', { error })
}
}

return profile
}

export const nextAuthOptions: AuthOptions = {
pages: {
signIn: '/auth/login',
Expand All @@ -124,36 +161,7 @@ export const nextAuthOptions: AuthOptions = {
url: `${apiBaseUrl}/user`,
// The built-in GitHub provider hardcodes `https://api.github.com/user/emails`
// for the email fallback. Override the request so we use the configured API host.
async request({ client, tokens }) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const profile = (await client.userinfo(tokens.access_token!)) as any

if (!profile.email) {
try {
const res = await fetch(`${apiBaseUrl}/user/emails`, {
headers: {
Authorization: `token ${tokens.access_token}`,
'User-Agent': 'private-mirrors-app',
},
})

if (res.ok) {
const emails: Array<{
email: string
primary: boolean
verified: boolean
}> = await res.json()
profile.email = (
emails.find((e) => e.primary) ?? emails[0]
)?.email
}
} catch (error) {
authLogger.warn('Failed to fetch user emails', { error })
}
}

return profile
},
request: createGitHubUserinfoRequest(apiBaseUrl),
},
}),
],
Expand Down
23 changes: 21 additions & 2 deletions src/bot/rest.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,27 @@
import { config } from '@probot/octokit-plugin-config'
import { Octokit as Core } from 'octokit'
import { getGitHubApiUrl } from '../utils/github-urls'
import { getGitHubApiUrl, getGitHubGraphQlUrl } from '../utils/github-urls'

export const Octokit = Core.plugin(config).defaults({
type GraphQlConfigurableOctokit = {
graphql: {
defaults: (options: {
url: string
}) => GraphQlConfigurableOctokit['graphql']
}
}

export const githubGraphQlEndpointPlugin = (octokit: unknown) => {
const graphQlCapableOctokit = octokit as GraphQlConfigurableOctokit
graphQlCapableOctokit.graphql = graphQlCapableOctokit.graphql.defaults({
url: getGitHubGraphQlUrl(),
})
return {}
}

export const Octokit = Core.plugin(
config,
githubGraphQlEndpointPlugin,
).defaults({
userAgent: `octokit-rest.js/repo-sync-bot`,
baseUrl: getGitHubApiUrl(),
})
Expand Down
5 changes: 4 additions & 1 deletion src/pages/api/webhooks.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
import app from 'bot'
import { createNodeMiddleware, createProbot, ProbotOctokit } from 'probot'
import { githubGraphQlEndpointPlugin } from 'bot/rest'
import { getGitHubApiUrl } from 'utils/github-urls'
import { logger } from 'utils/logger'

const baseUrl = getGitHubApiUrl()

// Configure Probot's Octokit with the GHE/GHES/github.com API base URL so
// every `context.octokit.*` call hits the correct host.
const GheProbotOctokit = ProbotOctokit.defaults({ baseUrl })
const GheProbotOctokit = ProbotOctokit.plugin(
githubGraphQlEndpointPlugin,
).defaults({ baseUrl })

export const probot = createProbot({ defaults: { Octokit: GheProbotOctokit } })

Expand Down
4 changes: 2 additions & 2 deletions src/server/git/controller.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import simpleGit, { SimpleGitOptions } from 'simple-git'
import { generateAuthUrl } from '../../utils/auth'
import { getCommitterEmailDomain } from '../../utils/github-urls'
import { temporaryDirectory } from 'tempy'
import { logger } from '../../utils/logger'
import { getCommitterEmailDomainWithWarning } from '../../utils/server/committer-email'
import { SyncReposSchema } from './schema'

const gitApiLogger = logger.getSubLogger({ name: 'git-api' })
Expand Down Expand Up @@ -59,7 +59,7 @@ export const syncReposHandler = async ({
const options: Partial<SimpleGitOptions> = {
config: [
`user.name=pma[bot]`,
`user.email=${input.source.octokit.installationId}+pma[bot]@${getCommitterEmailDomain()}`,
`user.email=${input.source.octokit.installationId}+pma[bot]@${getCommitterEmailDomainWithWarning()}`,
],
}

Expand Down
4 changes: 2 additions & 2 deletions src/server/repos/controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import simpleGit, { SimpleGitOptions } from 'simple-git'
import { generateAuthUrl } from 'utils/auth'
import { getCommitterEmailDomain } from 'utils/github-urls'
import { temporaryDirectory } from 'tempy'
import { getConfig } from '../../bot/config'
import {
Expand All @@ -12,6 +11,7 @@ import {
} from '../../bot/octokit'
import { Octokit } from '../../bot/rest'
import { logger } from '../../utils/logger'
import { getCommitterEmailDomainWithWarning } from '../../utils/server/committer-email'
import {
CreateMirrorSchema,
DeleteMirrorSchema,
Expand Down Expand Up @@ -222,7 +222,7 @@ export const createMirrorHandler = async ({
config: [
`user.name=pma[bot]`,
// We want to use the private installation ID as the email so that we can push to the private repo
`user.email=${privateInstallationId}+pma[bot]@${getCommitterEmailDomain()}`,
`user.email=${privateInstallationId}+pma[bot]@${getCommitterEmailDomainWithWarning()}`,
],
}
const git = simpleGit(tempDir, options)
Expand Down
4 changes: 2 additions & 2 deletions src/utils/auth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { TRPCError } from '@trpc/server'
import { getConfig } from '../bot/config'
import { personalOctokit } from '../bot/octokit'
import { logger } from '../utils/logger'
import { getGitHubServerHost } from './github-urls'
import { getGitHubServerHost, getGitHubServerProtocol } from './github-urls'

/**
* Generates a git url with the access token in it
Expand All @@ -19,7 +19,7 @@ export const generateAuthUrl = (
const USER = 'x-access-token'
const PASS = accessToken
const REPO = `${getGitHubServerHost()}/${owner}/${repo}`
return `https://${USER}:${PASS}@${REPO}`
return `${getGitHubServerProtocol()}//${USER}:${PASS}@${REPO}`
}

const middlewareLogger = logger.getSubLogger({ name: 'middleware' })
Expand Down
28 changes: 27 additions & 1 deletion src/utils/github-urls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
const DEFAULT_SERVER_URL = 'https://github.com'
const DEFAULT_API_URL = 'https://api.github.com'
const DEFAULT_EMAIL_DOMAIN = 'users.noreply.github.com'
const GHES_API_V3_SUFFIX_REGEX = /\/api\/v3\/?$/
const isGithubDotComHost = (host: string) =>
host === 'github.com' || host === 'www.github.com'

const stripTrailingSlash = (value: string) => value.replace(/\/+$/, '')

Expand All @@ -33,14 +36,17 @@ const safeUrl = (value: string | undefined | null): URL | null => {
* - `https://github.com` => `https://api.github.com`
* - `https://<tenant>.ghe.com` => `https://api.<tenant>.ghe.com`
* - anything else (GHES) => `<server>/api/v3`
*
* Keep this derivation in sync with the local fallback in
* `scripts/webhook-relay.mjs`.
*/
export const deriveApiUrlFromServerUrl = (serverUrl: string): string => {
const url = safeUrl(serverUrl)
if (!url) return DEFAULT_API_URL

const host = url.host.toLowerCase()

if (host === 'github.com' || host === 'www.github.com') {
if (isGithubDotComHost(host)) {
return DEFAULT_API_URL
}

Expand Down Expand Up @@ -76,6 +82,18 @@ export const getGitHubApiUrl = (): string => {
return stripTrailingSlash(deriveApiUrlFromServerUrl(getGitHubServerUrl()))
}

/**
* Returns the GraphQL endpoint URL (e.g. `https://api.github.com/graphql`).
* Safe to call from both server and client code.
*/
export const getGitHubGraphQlUrl = (): string => {
const apiUrl = getGitHubApiUrl()
if (GHES_API_V3_SUFFIX_REGEX.test(apiUrl)) {
return apiUrl.replace(GHES_API_V3_SUFFIX_REGEX, '/api/graphql')
}
return `${apiUrl}/graphql`
}

/**
* Returns the hostname portion of the GitHub server URL (e.g. `github.com`).
* Used to build authenticated git URLs.
Expand All @@ -84,6 +102,14 @@ export const getGitHubServerHost = (): string => {
return safeUrl(getGitHubServerUrl())?.host ?? 'github.com'
}

/**
* Returns the scheme portion of the GitHub server URL (e.g. `https:`).
* Used to build authenticated git URLs.
*/
export const getGitHubServerProtocol = (): string => {
return safeUrl(getGitHubServerUrl())?.protocol ?? 'https:'
}

/**
* Returns the OAuth authorize URL.
*/
Expand Down
31 changes: 31 additions & 0 deletions src/utils/server/committer-email.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { getCommitterEmailDomain, getGitHubServerUrl } from '../github-urls'
import { logger } from '../logger'

const githubUrlsLogger = logger.getSubLogger({ name: 'github-urls' })

let hasWarnedAboutDefaultCommitterEmailDomain = false

const isGithubDotComServer = (serverUrl: string) => {
try {
const host = new URL(serverUrl).host.toLowerCase()
return host === 'github.com' || host === 'www.github.com'
} catch {
return true
}
}

export const getCommitterEmailDomainWithWarning = () => {
if (
!hasWarnedAboutDefaultCommitterEmailDomain &&
!process.env.GITHUB_USER_EMAIL_DOMAIN &&
!isGithubDotComServer(getGitHubServerUrl())
) {
hasWarnedAboutDefaultCommitterEmailDomain = true
githubUrlsLogger.warn(
'GITHUB_USER_EMAIL_DOMAIN is not set for a non-github.com GitHub server; defaulting to users.noreply.github.com.',
{ serverUrl: getGitHubServerUrl() },
)
}

return getCommitterEmailDomain()
}
Loading