Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ PRIVATE_ORG=
# Leave unset for github.com. For GHE.com Data Residency, set GITHUB_SERVER_URL
# to your tenant URL (e.g. https://acme.ghe.com). For GHES, set it to your
# server URL (e.g. https://ghes.example.com). GITHUB_API_URL is derived
# automatically but can be overridden if needed.
# automatically but can be overridden if needed. GraphQL is derived from the
# REST API base and uses /api/graphql on GHES.
# NEXT_PUBLIC_* variants must also be set at build time (Docker build args) so
# client bundles and UI links target the correct host.
GITHUB_SERVER_URL=
Expand All @@ -42,7 +43,9 @@ NEXT_PUBLIC_GITHUB_API_URL=

# Committer email domain used on sync commits. Defaults to
# `users.noreply.github.com`. Set explicitly for GHE/GHES (the exact value
# depends on instance configuration).
# depends on instance configuration). If you leave it unset on a non-github.com
# deployment, the app logs a warning and still falls back to the github.com
# noreply domain for compatibility.
GITHUB_USER_EMAIL_DOMAIN=

# Used to skip branch protection creation if organization level branch protections are used instead
Expand Down
5 changes: 5 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@ WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

ARG NEXT_PUBLIC_GITHUB_SERVER_URL
ARG NEXT_PUBLIC_GITHUB_API_URL

ENV NEXT_TELEMETRY_DISABLED=1
ENV NEXT_PUBLIC_GITHUB_SERVER_URL=$NEXT_PUBLIC_GITHUB_SERVER_URL
ENV NEXT_PUBLIC_GITHUB_API_URL=$NEXT_PUBLIC_GITHUB_API_URL

RUN npm run build
RUN npm prune --omit=dev
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,11 @@ GITHUB_SERVER_URL=https://acme.ghe.com
# Required for client-side hooks and UI links to point at the correct host.
NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com

# Optional. Auto-derived from GITHUB_SERVER_URL:
# Optional REST API base URL. Auto-derived from GITHUB_SERVER_URL:
# github.com -> https://api.github.com
# <tenant>.ghe.com -> https://api.<tenant>.ghe.com
# <ghes-host> -> https://<ghes-host>/api/v3
# GraphQL is derived from this value and uses /api/graphql on GHES.
# Override only if the auto-derivation does not match your instance.
GITHUB_API_URL=
NEXT_PUBLIC_GITHUB_API_URL=
Expand All @@ -131,7 +132,8 @@ GITHUB_USER_EMAIL_DOMAIN=users.noreply.acme.ghe.com
Notes:

- The OAuth App / GitHub App, organizations, members and forks must all live on the same GHE instance.
- The `NEXT_PUBLIC_*` variables are inlined into the client bundle at build time. When building the Docker image, pass them as build args (e.g. `--build-arg NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com`) and update the `Dockerfile` to forward them into the `npm run build` step.
- The `NEXT_PUBLIC_*` variables are inlined into the client bundle at build time. When building the Docker image, pass them as build args (e.g. `--build-arg NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com`). The bundled `Dockerfile` already forwards them into the `npm run build` step.
- If you leave `GITHUB_USER_EMAIL_DOMAIN` unset on a non-github.com deployment, the app still falls back to `users.noreply.github.com` for compatibility, but it now logs a warning so you can correct the configuration.
- The local webhook relay (`npm run webhook`) uses `github-app-webhook-relay-polling` against the GitHub App hook deliveries endpoint. It is best-effort on GHE; in production, use real webhook deliveries configured directly on your GitHub App.

## Usage
Expand Down
2 changes: 1 addition & 1 deletion docs/developing.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ This will create an optimized production build of the app in the `out` directory

### Building for GHE.com / GHES

The `NEXT_PUBLIC_GITHUB_SERVER_URL` and `NEXT_PUBLIC_GITHUB_API_URL` env vars are inlined into the client bundle at build time. When targeting a GHE.com Data Residency tenant or a GHES instance, you must set them before running `npm run build` (or pass them as Docker build args). For example:
The `NEXT_PUBLIC_GITHUB_SERVER_URL` and `NEXT_PUBLIC_GITHUB_API_URL` env vars are inlined into the client bundle at build time. When targeting a GHE.com Data Residency tenant or a GHES instance, you must set them before running `npm run build` (or pass them as Docker build args). The bundled `Dockerfile` already forwards these build args into `npm run build`. For example:

```sh
NEXT_PUBLIC_GITHUB_SERVER_URL=https://acme.ghe.com \
Expand Down
3 changes: 3 additions & 0 deletions scripts/webhook-relay.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ if (!process.env.PUBLIC_ORG) {

const url = `${process.env.NEXTAUTH_URL}/api/webhooks`

// Keep this fallback in sync with deriveApiUrlFromServerUrl in
// src/utils/github-urls.ts. The relay only needs the REST API base URL; GraphQL
// callers must use /api/graphql on GHES.
const deriveApiUrl = (serverUrl) => {
try {
const u = new URL(serverUrl)
Expand Down
68 changes: 38 additions & 30 deletions src/app/api/auth/lib/nextauth-options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,43 @@ export const refreshAccessToken = async (

const apiBaseUrl = getGitHubApiUrl()

export const createGitHubUserinfoRequest =
(apiBaseUrl: string) =>
async ({
client,
tokens,
}: {
client: { userinfo: (accessToken: string) => Promise<unknown> }
tokens: { access_token?: string | null }
}) => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const profile = (await client.userinfo(tokens.access_token!)) as any

if (!profile.email) {
try {
const res = await fetch(`${apiBaseUrl}/user/emails`, {
headers: {
Authorization: `token ${tokens.access_token}`,
'User-Agent': 'private-mirrors-app',
},
})

if (res.ok) {
const emails: Array<{
email: string
primary: boolean
verified: boolean
}> = await res.json()
profile.email = (emails.find((e) => e.primary) ?? emails[0])?.email
}
} catch (error) {
authLogger.warn('Failed to fetch user emails', { error })
}
}

return profile
}

export const nextAuthOptions: AuthOptions = {
pages: {
signIn: '/auth/login',
Expand All @@ -124,36 +161,7 @@ export const nextAuthOptions: AuthOptions = {
url: `${apiBaseUrl}/user`,
// The built-in GitHub provider hardcodes `https://api.github.com/user/emails`
// for the email fallback. Override the request so we use the configured API host.
async request({ client, tokens }) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const profile = (await client.userinfo(tokens.access_token!)) as any

if (!profile.email) {
try {
const res = await fetch(`${apiBaseUrl}/user/emails`, {
headers: {
Authorization: `token ${tokens.access_token}`,
'User-Agent': 'private-mirrors-app',
},
})

if (res.ok) {
const emails: Array<{
email: string
primary: boolean
verified: boolean
}> = await res.json()
profile.email = (
emails.find((e) => e.primary) ?? emails[0]
)?.email
}
} catch (error) {
authLogger.warn('Failed to fetch user emails', { error })
}
}

return profile
},
request: createGitHubUserinfoRequest(apiBaseUrl),
},
}),
],
Expand Down
23 changes: 21 additions & 2 deletions src/bot/rest.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,27 @@
import { config } from '@probot/octokit-plugin-config'
import { Octokit as Core } from 'octokit'
import { getGitHubApiUrl } from '../utils/github-urls'
import { getGitHubApiUrl, getGitHubGraphQlUrl } from '../utils/github-urls'

export const Octokit = Core.plugin(config).defaults({
type GraphQlConfigurableOctokit = {
graphql: {
defaults: (options: {
url: string
}) => GraphQlConfigurableOctokit['graphql']
}
}

export const githubGraphQlEndpointPlugin = (octokit: unknown) => {
const graphQlCapableOctokit = octokit as GraphQlConfigurableOctokit
graphQlCapableOctokit.graphql = graphQlCapableOctokit.graphql.defaults({
url: getGitHubGraphQlUrl(),
})
return {}
}

export const Octokit = Core.plugin(
config,
githubGraphQlEndpointPlugin,
).defaults({
userAgent: `octokit-rest.js/repo-sync-bot`,
baseUrl: getGitHubApiUrl(),
})
Expand Down
5 changes: 4 additions & 1 deletion src/pages/api/webhooks.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
import app from 'bot'
import { createNodeMiddleware, createProbot, ProbotOctokit } from 'probot'
import { githubGraphQlEndpointPlugin } from 'bot/rest'
import { getGitHubApiUrl } from 'utils/github-urls'
import { logger } from 'utils/logger'

const baseUrl = getGitHubApiUrl()

// Configure Probot's Octokit with the GHE/GHES/github.com API base URL so
// every `context.octokit.*` call hits the correct host.
const GheProbotOctokit = ProbotOctokit.defaults({ baseUrl })
const GheProbotOctokit = ProbotOctokit.plugin(
githubGraphQlEndpointPlugin,
).defaults({ baseUrl })

export const probot = createProbot({ defaults: { Octokit: GheProbotOctokit } })

Expand Down
4 changes: 2 additions & 2 deletions src/server/git/controller.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import simpleGit, { SimpleGitOptions } from 'simple-git'
import { generateAuthUrl } from '../../utils/auth'
import { getCommitterEmailDomain } from '../../utils/github-urls'
import { temporaryDirectory } from 'tempy'
import { logger } from '../../utils/logger'
import { getCommitterEmailDomainWithWarning } from '../../utils/server/committer-email'
import { SyncReposSchema } from './schema'

const gitApiLogger = logger.getSubLogger({ name: 'git-api' })
Expand Down Expand Up @@ -59,7 +59,7 @@ export const syncReposHandler = async ({
const options: Partial<SimpleGitOptions> = {
config: [
`user.name=pma[bot]`,
`user.email=${input.source.octokit.installationId}+pma[bot]@${getCommitterEmailDomain()}`,
`user.email=${input.source.octokit.installationId}+pma[bot]@${getCommitterEmailDomainWithWarning()}`,
],
}

Expand Down
4 changes: 2 additions & 2 deletions src/server/repos/controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import simpleGit, { SimpleGitOptions } from 'simple-git'
import { generateAuthUrl } from 'utils/auth'
import { getCommitterEmailDomain } from 'utils/github-urls'
import { temporaryDirectory } from 'tempy'
import { getConfig } from '../../bot/config'
import {
Expand All @@ -12,6 +11,7 @@ import {
} from '../../bot/octokit'
import { Octokit } from '../../bot/rest'
import { logger } from '../../utils/logger'
import { getCommitterEmailDomainWithWarning } from '../../utils/server/committer-email'
import {
CreateMirrorSchema,
DeleteMirrorSchema,
Expand Down Expand Up @@ -222,7 +222,7 @@ export const createMirrorHandler = async ({
config: [
`user.name=pma[bot]`,
// We want to use the private installation ID as the email so that we can push to the private repo
`user.email=${privateInstallationId}+pma[bot]@${getCommitterEmailDomain()}`,
`user.email=${privateInstallationId}+pma[bot]@${getCommitterEmailDomainWithWarning()}`,
],
}
const git = simpleGit(tempDir, options)
Expand Down
4 changes: 2 additions & 2 deletions src/utils/auth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { TRPCError } from '@trpc/server'
import { getConfig } from '../bot/config'
import { personalOctokit } from '../bot/octokit'
import { logger } from '../utils/logger'
import { getGitHubServerHost } from './github-urls'
import { getGitHubServerHost, getGitHubServerProtocol } from './github-urls'

/**
* Generates a git url with the access token in it
Expand All @@ -19,7 +19,7 @@ export const generateAuthUrl = (
const USER = 'x-access-token'
const PASS = accessToken
const REPO = `${getGitHubServerHost()}/${owner}/${repo}`
return `https://${USER}:${PASS}@${REPO}`
return `${getGitHubServerProtocol()}//${USER}:${PASS}@${REPO}`
}

const middlewareLogger = logger.getSubLogger({ name: 'middleware' })
Expand Down
28 changes: 27 additions & 1 deletion src/utils/github-urls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
const DEFAULT_SERVER_URL = 'https://github.com'
const DEFAULT_API_URL = 'https://api.github.com'
const DEFAULT_EMAIL_DOMAIN = 'users.noreply.github.com'
const GHES_API_V3_SUFFIX_REGEX = /\/api\/v3\/?$/
export const isGithubDotComHost = (host: string) =>
host === 'github.com' || host === 'www.github.com'

const stripTrailingSlash = (value: string) => value.replace(/\/+$/, '')

Expand All @@ -33,14 +36,17 @@ const safeUrl = (value: string | undefined | null): URL | null => {
* - `https://github.com` => `https://api.github.com`
* - `https://<tenant>.ghe.com` => `https://api.<tenant>.ghe.com`
* - anything else (GHES) => `<server>/api/v3`
*
* Keep this derivation in sync with the local fallback in
* `scripts/webhook-relay.mjs`.
*/
export const deriveApiUrlFromServerUrl = (serverUrl: string): string => {
const url = safeUrl(serverUrl)
if (!url) return DEFAULT_API_URL

const host = url.host.toLowerCase()

if (host === 'github.com' || host === 'www.github.com') {
if (isGithubDotComHost(host)) {
return DEFAULT_API_URL
}

Expand Down Expand Up @@ -76,6 +82,18 @@ export const getGitHubApiUrl = (): string => {
return stripTrailingSlash(deriveApiUrlFromServerUrl(getGitHubServerUrl()))
}

/**
* Returns the GraphQL endpoint URL (e.g. `https://api.github.com/graphql`).
* Safe to call from both server and client code.
*/
export const getGitHubGraphQlUrl = (): string => {
const apiUrl = getGitHubApiUrl()
if (GHES_API_V3_SUFFIX_REGEX.test(apiUrl)) {
return apiUrl.replace(GHES_API_V3_SUFFIX_REGEX, '/api/graphql')
}
return `${apiUrl}/graphql`
}

/**
* Returns the hostname portion of the GitHub server URL (e.g. `github.com`).
* Used to build authenticated git URLs.
Expand All @@ -84,6 +102,14 @@ export const getGitHubServerHost = (): string => {
return safeUrl(getGitHubServerUrl())?.host ?? 'github.com'
}

/**
* Returns the scheme portion of the GitHub server URL (e.g. `https:`).
* Used to build authenticated git URLs.
*/
export const getGitHubServerProtocol = (): string => {
return safeUrl(getGitHubServerUrl())?.protocol ?? 'https:'
}

/**
* Returns the OAuth authorize URL.
*/
Expand Down
35 changes: 35 additions & 0 deletions src/utils/server/committer-email.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import {
getCommitterEmailDomain,
getGitHubServerUrl,
isGithubDotComHost,
} from '../github-urls'
import { logger } from '../logger'

const githubUrlsLogger = logger.getSubLogger({ name: 'github-urls' })

let hasWarnedAboutDefaultCommitterEmailDomain = false

const isGithubDotComServer = (serverUrl: string) => {
try {
const host = new URL(serverUrl).host.toLowerCase()
return isGithubDotComHost(host)
} catch {
return true
}
}

export const getCommitterEmailDomainWithWarning = () => {
if (
!hasWarnedAboutDefaultCommitterEmailDomain &&
!process.env.GITHUB_USER_EMAIL_DOMAIN &&
!isGithubDotComServer(getGitHubServerUrl())
) {
hasWarnedAboutDefaultCommitterEmailDomain = true
githubUrlsLogger.warn(
'GITHUB_USER_EMAIL_DOMAIN is not set for a non-github.com GitHub server; defaulting to users.noreply.github.com.',
{ serverUrl: getGitHubServerUrl() },
)
}

return getCommitterEmailDomain()
}
Loading