Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions docs/Salesforce-batchsink.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,27 @@ You also can use the macro function ${conn(connection-name)}.

**Reference Name:** Name used to uniquely identify this sink for lineage, annotating metadata, etc.

**Username:** Salesforce username.
**Grant Type:** Grant type to use for OAuth authentication. Supported values are 'password' and
'client_credentials'. When set to 'client_credentials', only Consumer Key, Consumer Secret, and Login URL
are required. Username, Password, and Security Token are not needed. Defaults to 'password' if not specified.

**Password:** Salesforce password.
**Username:** Salesforce username. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token, the plugin
will append the token before authenticating with Salesforce.
**Password:** Salesforce password. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token, the plugin
will append the token before authenticating with Salesforce. Only applicable for 'password' grant type.

**Consumer Key:** Application Consumer Key. This is also known as the OAuth client ID.
A Salesforce connected application must be created in order to get a consumer key.

**Consumer Secret:** Application Consumer Secret. This is also known as the OAuth client secret.
A Salesforce connected application must be created in order to get a client secret.

**Login URL:** Salesforce OAuth2 login URL.
**Login URL:** Salesforce OAuth2 login URL. For the 'password' grant type, the default generic URL
`https://login.salesforce.com/services/oauth2/token` can be used. For the 'client_credentials' grant type,
you must provide your Salesforce instance-specific URL, for example
`https://<your-instance>.my.salesforce.com/services/oauth2/token`.

**Connect Timeout:** Maximum time in milliseconds to wait for connection initialization before it times out.

Expand Down
17 changes: 12 additions & 5 deletions docs/Salesforce-batchsource.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,27 @@ Configuration

**Reference Name:** Name used to uniquely identify this source for lineage, annotating metadata, etc.

**Username:** Salesforce username.
**Grant Type:** Grant type to use for OAuth authentication. Supported values are 'password' and
'client_credentials'. When set to 'client_credentials', only Consumer Key, Consumer Secret, and Login URL
are required. Username, Password, and Security Token are not needed. Defaults to 'password' if not specified.

**Password:** Salesforce password.
**Username:** Salesforce username. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token the plugin,
will append the token before authenticating with Salesforce.
**Password:** Salesforce password. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token the plugin,
will append the token before authenticating with Salesforce. Only applicable for 'password' grant type.

**Consumer Key:** Application Consumer Key. This is also known as the OAuth client ID.
A Salesforce connected application must be created in order to get a consumer key.

**Consumer Secret:** Application Consumer Secret. This is also known as the OAuth client secret.
A Salesforce connected application must be created in order to get a client secret.

**Login URL:** Salesforce OAuth2 login URL.
**Login URL:** Salesforce OAuth2 login URL. For the 'password' grant type, the default generic URL
`https://login.salesforce.com/services/oauth2/token` can be used. For the 'client_credentials' grant type,
you must provide your Salesforce instance-specific URL, for example
`https://<your-instance>.my.salesforce.com/services/oauth2/token`.

**Connect Timeout:** Maximum time in milliseconds to wait for connection initialization before it times out.

Expand Down
15 changes: 11 additions & 4 deletions docs/Salesforce-connector.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,27 @@ Properties

**Description:** Description of the connection.

**Username:** Salesforce username.
**Grant Type:** Grant type to use for OAuth authentication. Supported values are 'password' and
'client_credentials'. When set to 'client_credentials', only Consumer Key, Consumer Secret, and Login URL
are required. Username, Password, and Security Token are not needed. Defaults to 'password' if not specified.

**Password:** Salesforce password.
**Username:** Salesforce username. Required for 'password' grant type.

**Password:** Salesforce password. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token, the plugin
will append the token before authenticating with Salesforce.
will append the token before authenticating with Salesforce. Only applicable for 'password' grant type.

**Consumer Key:** Application Consumer Key. This is also known as the OAuth client ID.
A Salesforce connected application must be created in order to get a consumer key.

**Consumer Secret:** Application Consumer Secret. This is also known as the OAuth client secret.
A Salesforce connected application must be created in order to get a client secret.

**Login URL:** Salesforce OAuth2 login URL.
**Login URL:** Salesforce OAuth2 login URL. For the 'password' grant type, the default generic URL
`https://login.salesforce.com/services/oauth2/token` can be used. For the 'client_credentials' grant type,
you must provide your Salesforce instance-specific URL, for example
`https://<your-instance>.my.salesforce.com/services/oauth2/token`.

**Connect Timeout:** Maximum time in milliseconds to wait for connection initialization before it times out.

Expand Down
17 changes: 12 additions & 5 deletions docs/Salesforce-streamingsource.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,27 @@ Configuration

**Reference Name:** Name used to uniquely identify this source for lineage, annotating metadata, etc.

**Username:** Salesforce username.
**Grant Type:** Grant type to use for OAuth authentication. Supported values are 'password' and
'client_credentials'. When set to 'client_credentials', only Consumer Key, Consumer Secret, and Login URL
are required. Username, Password, and Security Token are not needed. Defaults to 'password' if not specified.

**Password:** Salesforce password.
**Username:** Salesforce username. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token, the plugin
will append the token before authenticating with Salesforce.
**Password:** Salesforce password. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token, the plugin
will append the token before authenticating with Salesforce. Only applicable for 'password' grant type.

**Consumer Key:** Application Consumer Key. This is also known as the OAuth client ID.
A Salesforce connected application must be created in order to get a consumer key.

**Consumer Secret:** Application Consumer Secret. This is also known as the OAuth client secret.
A Salesforce connected application must be created in order to get a client secret.

**Login URL:** Salesforce OAuth2 login URL.
**Login URL:** Salesforce OAuth2 login URL. For the 'password' grant type, the default generic URL
`https://login.salesforce.com/services/oauth2/token` can be used. For the 'client_credentials' grant type,
you must provide your Salesforce instance-specific URL, for example
`https://<your-instance>.my.salesforce.com/services/oauth2/token`.

**Connect Timeout:** Maximum time in milliseconds to wait for connection initialization before it times out.

Expand Down
17 changes: 12 additions & 5 deletions docs/SalesforceMultiObjects-batchsource.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,27 @@ Configuration

**Reference Name:** Name used to uniquely identify this source for lineage, annotating metadata, etc.

**Username:** Salesforce username.
**Grant Type:** Grant type to use for OAuth authentication. Supported values are 'password' and
'client_credentials'. When set to 'client_credentials', only Consumer Key, Consumer Secret, and Login URL
are required. Username, Password, and Security Token are not needed. Defaults to 'password' if not specified.

**Password:** Salesforce password.
**Username:** Salesforce username. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token, the plugin
will append the token before authenticating with Salesforce.
**Password:** Salesforce password. Required for 'password' grant type.

**Security Token:** Salesforce security token. If the password does not contain the security token, the plugin
will append the token before authenticating with Salesforce. Only applicable for 'password' grant type.

**Consumer Key:** Application Consumer Key. This is also known as the OAuth client ID.
A Salesforce connected application must be created in order to get a consumer key.

**Consumer Secret:** Application Consumer Secret. This is also known as the OAuth client secret.
A Salesforce connected application must be created in order to get a client secret.

**Login URL:** Salesforce OAuth2 login URL.
**Login URL:** Salesforce OAuth2 login URL. For the 'password' grant type, the default generic URL
`https://login.salesforce.com/services/oauth2/token` can be used. For the 'client_credentials' grant type,
you must provide your Salesforce instance-specific URL, for example
`https://<your-instance>.my.salesforce.com/services/oauth2/token`.

**Connect Timeout:** Maximum time in milliseconds to wait for connection initialization before time out.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ public static OAuthInfo getOAuthInfo(SalesforceConnectorInfo config, FailureColl
if (!config.canAttemptToEstablishConnection()) {
return null;
}
config.validateAuthenticationFields(collector);
OAuthInfo oAuthInfo = null;
try {
oAuthInfo = Authenticator.getOAuthInfo(config.getAuthenticatorCredentials());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
package io.cdap.plugin.salesforce;

import io.cdap.cdap.api.plugin.PluginConfig;
import io.cdap.plugin.salesforce.authenticator.AuthenticatorCredentials.GrantType;

import java.util.Arrays;
import java.util.Collections;
Expand Down Expand Up @@ -65,11 +66,15 @@ public class SalesforceConstants {
public static final String CONFIG_MAX_RETRY_DURATION = "mapred.salesforce.maxRetryDuration";
public static final String CONFIG_MAX_RETRY_COUNT = "mapred.salesforce.maxRetryCount";
public static final String CONFIG_RETRY_REQUIRED = "mapred.salesforce.retryOnBackendError";
public static final String CONFIG_GRANT_TYPE = "mapred.salesforce.grantType";

public static final String PROPERTY_PROXY_URL = "proxyUrl";
public static final String CONFIG_PROXY_URL = "mapred.salesforce.proxyUrl";
public static final String REGEX_PROXY_URL = "^(?i)(https?)://.*$";

public static final String PROPERTY_AUTHENTICATION_GRANT_TYPE = "authenticationGrantType";
public static final GrantType DEFAULT_GRANT_TYPE = GrantType.PASSWORD;

public static final String PROPERTY_MAX_RETRY_TIME_IN_MINS = "cdap.streaming.maxRetryTimeInMins";
public static final long DEFAULT_MAX_RETRY_TIME_IN_MINS = 360L;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
*/
package io.cdap.plugin.salesforce.plugin;

import com.google.common.base.Strings;
import com.sforce.ws.ConnectionException;
import io.cdap.cdap.api.annotation.Description;
import io.cdap.cdap.api.annotation.Macro;
Expand All @@ -24,6 +25,7 @@
import io.cdap.plugin.salesforce.SalesforceConnectionUtil;
import io.cdap.plugin.salesforce.SalesforceConstants;
import io.cdap.plugin.salesforce.authenticator.AuthenticatorCredentials;
import io.cdap.plugin.salesforce.authenticator.AuthenticatorCredentials.GrantType;

import javax.annotation.Nullable;

Expand All @@ -32,6 +34,12 @@
*/
public class SalesforceConnectorBaseConfig extends PluginConfig {

@Name(SalesforceConstants.PROPERTY_AUTHENTICATION_GRANT_TYPE)
@Description("Salesforce authentication grant type: basic or client credentials")
@Nullable
@Macro
protected String authenticationGrantType;

@Nullable
@Name(SalesforceConstants.PROPERTY_PROXY_URL)
@Description("Proxy URL. Must contain a protocol, address and port.")
Expand Down Expand Up @@ -118,7 +126,8 @@ public SalesforceConnectorBaseConfig(@Nullable String consumerKey,
@Nullable Long initialRetryDuration,
@Nullable Long maxRetryDuration,
@Nullable Integer maxRetryCount,
@Nullable Boolean retryOnBackendError) {
@Nullable Boolean retryOnBackendError,
@Nullable String authenticationGrantType) {
this.consumerKey = consumerKey;
this.consumerSecret = consumerSecret;
this.username = username;
Expand All @@ -132,6 +141,16 @@ public SalesforceConnectorBaseConfig(@Nullable String consumerKey,
this.maxRetryDuration = maxRetryDuration;
this.retryOnBackendError = retryOnBackendError;
this.maxRetryCount = maxRetryCount;
this.authenticationGrantType = authenticationGrantType;
}

public GrantType getAuthenticationGrantType() {
if (!Strings.isNullOrEmpty(authenticationGrantType) &&
authenticationGrantType.equals(GrantType.CLIENT_CREDENTIALS.getType())) {
return GrantType.CLIENT_CREDENTIALS;
}
// Default auth, handles null case when upgrading pipeline
return SalesforceConstants.DEFAULT_GRANT_TYPE;
}

@Nullable
Expand Down Expand Up @@ -232,4 +251,55 @@ public String getProxyUrl() {
return proxyUrl;
}

/**
* Validates that required authentication fields are present based on the selected OAuth grant type.
* For PASSWORD grant type: consumerKey, consumerSecret, username, password, and loginUrl are required.
* For CLIENT_CREDENTIALS grant type: consumerKey, consumerSecret, and loginUrl are required.
*
* @param collector the failure collector to report validation errors
*/
public void validateAuthenticationFields(FailureCollector collector) {
if (containsMacro(SalesforceConstants.PROPERTY_AUTHENTICATION_GRANT_TYPE)) {
return;
}

GrantType grantType = getAuthenticationGrantType();

// Fields required for all grant types
if (!containsMacro(SalesforceConstants.PROPERTY_CONSUMER_KEY) && Strings.isNullOrEmpty(consumerKey)) {
collector.addFailure("Consumer Key is required for authentication.",
"Please provide the Consumer Key from your Salesforce connected app.")
.withConfigProperty(SalesforceConstants.PROPERTY_CONSUMER_KEY);
}
if (!containsMacro(SalesforceConstants.PROPERTY_CONSUMER_SECRET) && Strings.isNullOrEmpty(consumerSecret)) {
collector.addFailure("Consumer Secret is required for authentication.",
"Please provide the Consumer Secret from your Salesforce connected app.")
.withConfigProperty(SalesforceConstants.PROPERTY_CONSUMER_SECRET);
}
if (!containsMacro(SalesforceConstants.PROPERTY_LOGIN_URL) && Strings.isNullOrEmpty(loginUrl)) {
collector.addFailure("Login URL is required for authentication.",
"Please provide the Salesforce login URL.")
.withConfigProperty(SalesforceConstants.PROPERTY_LOGIN_URL);
}

// Fields required only for PASSWORD grant type
if (grantType == GrantType.PASSWORD) {
if (!containsMacro(SalesforceConstants.PROPERTY_USERNAME) && Strings.isNullOrEmpty(username)) {
collector.addFailure("Username is required for password grant type authentication.",
"Please provide the Salesforce username.")
.withConfigProperty(SalesforceConstants.PROPERTY_USERNAME);
}
if (!containsMacro(SalesforceConstants.PROPERTY_PASSWORD) && Strings.isNullOrEmpty(password)) {
collector.addFailure("Password is required for password grant type authentication.",
"Please provide the Salesforce password.")
.withConfigProperty(SalesforceConstants.PROPERTY_PASSWORD);
}
if (!containsMacro(SalesforceConstants.PROPERTY_SECURITY_TOKEN) && Strings.isNullOrEmpty(securityToken)) {
collector.addFailure("Security Token is required for password grant type authentication.",
"Please provide the Salesforce security token.")
.withConfigProperty(SalesforceConstants.PROPERTY_SECURITY_TOKEN);
}
}
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
import io.cdap.plugin.salesforce.SalesforceConnectionUtil;
import io.cdap.plugin.salesforce.SalesforceConstants;
import io.cdap.plugin.salesforce.authenticator.AuthenticatorCredentials;
import io.cdap.plugin.salesforce.authenticator.AuthenticatorCredentials.GrantType;
import io.cdap.plugin.salesforce.plugin.connector.SalesforceConnectorConfig;

import javax.annotation.Nullable;
Expand Down Expand Up @@ -103,6 +104,14 @@ public Boolean isRetryOnBackendError() {
return config.isRetryOnBackendError();
}

public GrantType getAuthenticationGrantType() {
return config.getAuthenticationGrantType();
}

public void validateAuthenticationFields(FailureCollector collector) {
config.validateAuthenticationFields(collector);
}

public void validate(FailureCollector collector, @Nullable OAuthInfo oAuthInfo) {
try {
validateConnection(oAuthInfo);
Expand Down Expand Up @@ -149,6 +158,7 @@ public boolean canAttemptToEstablishConnection() {

return !(config.containsMacro(SalesforceConstants.PROPERTY_CONSUMER_KEY)
|| config.containsMacro(SalesforceConstants.PROPERTY_CONSUMER_SECRET)
|| config.containsMacro(SalesforceConstants.PROPERTY_AUTHENTICATION_GRANT_TYPE)
|| config.containsMacro(SalesforceConstants.PROPERTY_USERNAME)
|| config.containsMacro(SalesforceConstants.PROPERTY_PASSWORD)
|| config.containsMacro(SalesforceConstants.PROPERTY_LOGIN_URL)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,10 @@ public SalesforceConnectorConfig(@Nullable String consumerKey,
@Nullable Long initialRetryDuration,
@Nullable Long maxRetryDuration,
@Nullable Integer maxRetryCount,
@Nullable Boolean retryOnBackendError) {
@Nullable Boolean retryOnBackendError,
@Nullable String authenticationGrantType) {
super(consumerKey, consumerSecret, username, password, loginUrl, securityToken, connectTimeout, readTimeout,
proxyUrl, initialRetryDuration, maxRetryDuration, maxRetryCount, retryOnBackendError);
proxyUrl, initialRetryDuration, maxRetryDuration, maxRetryCount, retryOnBackendError, authenticationGrantType);
this.oAuthInfo = oAuthInfo;
}

Expand Down
Loading
Loading