Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,10 @@
* or results in an empty queue (takes the last element).
*
* <p>Note that this model generally assumes that <i>false positives</i> are okay, meaning that the
* availability future completes despite there being no data availabile in the queue. The consumer
* is responsible for polling data and obtaining another future to wait on. This is similar to the
* way that Java's Monitors and Conditions can have the <i>spurious wakeup</i> of the waiting
* threads and commonly need to be used in loop with the waiting condition.
* availability future completes despite there being no data available in the queue. The consumer is
* responsible for polling data and obtaining another future to wait on. This is similar to the way
* that Java's Monitors and Conditions can have the <i>spurious wakeup</i> of the waiting threads
* and commonly need to be used in loop with the waiting condition.
*
* <h3>Producer Wakeup</h3>
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
/**
* A {@link SinkWriter} implementation for {@link FileSink}.
*
* <p>It writes data to and manages the different active {@link FileWriterBucket buckes} in the
* <p>It writes data to and manages the different active {@link FileWriterBucket buckets} in the
* {@link FileSink}.
*
* @param <IN> The type of input elements.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
/**
* Partition commit policy to update metastore.
*
* <p>If this is for file system table, the metastore is a empty implemantation. If this is for hive
* table, the metastore is for connecting to hive metastore.
* <p>If this is for file system table, the metastore is an empty implementation. If this is for
* hive table, the metastore is for connecting to hive metastore.
*/
@Internal
public class MetastoreCommitPolicy implements PartitionCommitPolicy {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ interface Context<P> extends Serializable {
* A comparable partition value that can compare order by using its comparator.
*
* @param <P> The type of partition value.
* @param <T> The tye of the partition offset, the partition offset is comparable.
* @param <T> The type of the partition offset, the partition offset is comparable.
*/
interface ComparablePartitionValue<P, T extends Comparable<T>> extends Serializable {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ enum RedistributionMode {
REDISTRIBUTABLE,

/**
* States are guranteed to be identical in different partitions, thus redistribution is not
* States are guaranteed to be identical in different partitions, thus redistribution is not
* a problem.
*/
IDENTICAL
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ public static void initDefaultsFromConfiguration(Configuration configuration) {
protected transient FSDataOutputStream stream;

/**
* The path that is actually written to (may a a file in a the directory defined by {@code
* The path that is actually written to (may be a file in the directory defined by {@code
* outputFilePath} )
*/
private transient Path actualFilePath;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ public class SinkUtils implements Serializable {
/**
* Acquire permits on the given semaphore within a given allowed timeout and deal with errors.
*
* @param permits the mumber of permits to acquire.
* @param permits the number of permits to acquire.
* @param maxConcurrentRequests the maximum number of permits the semaphore was initialized
* with.
* @param maxConcurrentRequestsTimeout the timeout to acquire the permits.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,7 @@ void execute(

/**
* Return if operator/function should interrupt a longer computation and return from the
* currently processed elemenent/watermark, for example in order to let Flink perform a
* currently processed element/watermark, for example in order to let Flink perform a
* checkpoint.
*
* @return whether operator/function should interrupt its computation.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ public boolean supportsCompareAgainstReference() {
*
* @param first The first record.
* @param second The second record.
* @return An integer defining the oder among the objects in the same way as {@link
* @return An integer defining the order among the objects in the same way as {@link
* java.util.Comparator#compare(Object, Object)}.
* @see java.util.Comparator#compare(Object, Object)
*/
Expand All @@ -169,7 +169,7 @@ public boolean supportsCompareAgainstReference() {
*
* @param firstSource The input view containing the first record.
* @param secondSource The input view containing the second record.
* @return An integer defining the oder among the objects in the same way as {@link
* @return An integer defining the order among the objects in the same way as {@link
* java.util.Comparator#compare(Object, Object)}.
* @throws IOException Thrown, if any of the input views raised an exception when reading the
* records.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -467,7 +467,7 @@ private static RowKind readKindFromMask(boolean[] mask) {
}

// --------------------------------------------------------------------------------------------
// Serializer configuration snapshoting & compatibility
// Serializer configuration snapshotting & compatibility
// --------------------------------------------------------------------------------------------

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ private static void logConfiguration(
/**
* Flattens a nested configuration map to be only one level deep.
*
* <p>Nested keys are concatinated using the {@code KEY_SEPARATOR} character. So that:
* <p>Nested keys are concatenated using the {@code KEY_SEPARATOR} character. So that:
*
* <pre>
* keyA:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
* subset of the whole union state. On restore, each subtask's restored union state should have one
* entry for each subtask in the previous run.
*
* <p>All input elements to the operator arre simply passed through a user-provided map function and
* <p>All input elements to the operator are simply passed through a user-provided map function and
* emitted.
*/
public class ArtificalOperatorStateMapper<IN, OUT> extends RichMapFunction<IN, OUT>
Expand Down
2 changes: 1 addition & 1 deletion flink-end-to-end-tests/run-single-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
# limitations under the License.
################################################################################

# This can be used to run a single test in the context of a test runnner
# This can be used to run a single test in the context of a test runner
# Usage: ./run-single-test.sh [skip] test-scripts/my-test-case.sh

# IMPORTANT:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ under the License.
<value>file:///hadoop-data/nm-log-dirs</value>
</property>
<!--
This allows running the tests also in low disk space environments. YARN is quite aggresive with
This allows running the tests also in low disk space environments. YARN is quite aggressive with
blacklisting nodes on low memory: https://issues.apache.org/jira/browse/FLINK-16136
-->
<property>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ function run_resume_externalized_checkpoints() {
wait_job_running $DATASTREAM_JOB
wait_oper_metric_num_in_records SemanticsCheckMapper.0 200

# if state is errorneous and the general purpose DataStream job produces alerting messages,
# if state is erroneous and the general purpose DataStream job produces alerting messages,
# output would be non-empty and the test will not pass
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ run_resume_savepoint_test() {

wait_oper_metric_num_in_records SemanticsCheckMapper.0 200

# if state is errorneous and the state machine job produces alerting state transitions,
# if state is erroneous and the state machine job produces alerting state transitions,
# output would be non-empty and the test will not pass
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
/**
* A RecoverableFsDataOutputStream to OSS that is based on a recoverable multipart upload.
*
* <p>This class is NOT thread-safe. Concurrent writes tho this stream result in corrupt or lost
* <p>This class is NOT thread-safe. Concurrent writes to this stream result in corrupt or lost
* data.
*
* <p>The {@link #close()} method may be called concurrently when cancelling / shutting down. It
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
/**
* A RecoverableFsDataOutputStream to S3 that is based on a recoverable multipart upload.
*
* <p>This class is NOT thread-safe. Concurrent writes tho this stream result in corrupt or lost
* <p>This class is NOT thread-safe. Concurrent writes to this stream result in corrupt or lost
* data.
*
* <p>The {@link #close()} method may be called concurrently when cancelling / shutting down. It
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
import java.util.Optional;
import java.util.concurrent.Executor;

/** A factory for creating or recovering {@link RecoverableMultiPartUpload mulitpart uploads}. */
/** A factory for creating or recovering {@link RecoverableMultiPartUpload multipart uploads}. */
@Internal
final class S3RecoverableMultipartUploadFactory {

Expand Down
2 changes: 1 addition & 1 deletion flink-formats/flink-avro/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ under the License.
</execution>
</executions>
</plugin>
<!--Remove the AvroExternalJarProgram code from the test-classes directory since it musn't be in the
<!--Remove the AvroExternalJarProgram code from the test-classes directory since it mustn't be in the
classpath when running the tests to actually test whether the user code class loader
is properly used.-->
<plugin>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
import java.util.List;

/**
* Converter functions that covert Flink's type information to Jackson's {@link CsvSchema}.
* Converter functions that convert Flink's type information to Jackson's {@link CsvSchema}.
*
* <p>In {@link CsvSchema}, there are four types (string, number, boolean, and array). In order to
* satisfy various Flink types, this class sorts out instances of {@link TypeInformation} and {@link
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
* </ol>
*
* <p>Each {@link Pattern} can be optional and have a {@link ConsumingStrategy}. Looping and Times
* also hava an additional inner consuming strategy that is applied between accepted events in the
* also have an additional inner consuming strategy that is applied between accepted events in the
* pattern.
*/
public class Quantifier {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@

/**
* A {@code SavepointWriter} can create new savepoints from bounded data streams. This can allow for
* boostrapping state for new applications or modifying the savepoints of existing jobs.
* bootstrapping state for new applications or modifying the savepoints of existing jobs.
*/
@PublicEvolving
public class SavepointWriter {
Expand Down
2 changes: 1 addition & 1 deletion flink-python/docs/reference/pyflink.datastream/state.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ etc).

The :class:`EmbeddedRocksDBStateBackend` stores working state in an embedded
`RocksDB <http://rocksdb.org/>`_, instance and is able to scale working state to many
terrabytes in size, only limited by available disk space across all task managers.
terabytes in size, only limited by available disk space across all task managers.

**Raw Bytes Storage and Backends**

Expand Down
4 changes: 2 additions & 2 deletions flink-python/pyflink/datastream/checkpoint_storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ class FileSystemCheckpointStorage(CheckpointStorage):
many small files. The threshold for that is configurable. When increasing this threshold, the
size of the checkpoint metadata increases. The checkpoint metadata of all retained completed
checkpoints needs to fit into the JobManager's heap memory. This is typically not a problem,
unless the threashold `get_min_file_size_threshold` is increased significantly.
unless the threshold `get_min_file_size_threshold` is increased significantly.

**Persistence Guarantees**

Expand Down Expand Up @@ -340,7 +340,7 @@ def get_min_file_size_threshold(self) -> int:
"""
Gets the threshold below which state is stored as part of the metadata, rather than in
file. This threshold ensures the backend does not create a large amount of small files,
where potentially the file pointers are larget than the state itself.
where potentially the file pointers are larger than the state itself.
"""
return self._j_checkpoint_storage.getMinFileSizeThreshold()

Expand Down
4 changes: 2 additions & 2 deletions flink-python/pyflink/datastream/connectors/kafka.py
Original file line number Diff line number Diff line change
Expand Up @@ -256,8 +256,8 @@ class FlinkKafkaProducerBase(SinkFunction, ABC):
Flink Sink to produce data into a Kafka topic.

Please note that this producer provides at-least-once reliability guarantees when checkpoints
are enabled and set_flush_on_checkpoint(True) is set. Otherwise, the producer doesn;t provid any
reliability guarantees.
are enabled and set_flush_on_checkpoint(True) is set. Otherwise, the producer doesn't provide
any reliability guarantees.
"""

def __init__(self, j_flink_kafka_producer):
Expand Down
4 changes: 2 additions & 2 deletions flink-python/pyflink/datastream/functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ def get_list_state(self, state_descriptor: ListStateDescriptor) -> ListState:
"""
Gets a handle to the system's key/value list state. This state is similar to the value state
access, but is optimized for state that holds lists. One can add elements to the list, or
retrieve the list as a whle.
retrieve the list as a whole.

This state is only accessible if the function is executed on a KeyedStream.
"""
Expand Down Expand Up @@ -288,7 +288,7 @@ class FlatMapFunction(Function):
@abstractmethod
def flat_map(self, value):
"""
The core mthod of the FlatMapFunction. Takes an element from the input data and transforms
The core method of the FlatMapFunction. Takes an element from the input data and transforms
it into zero, one, or more elements.
A basic implementation of flat map is as follows:

Expand Down
4 changes: 2 additions & 2 deletions flink-python/pyflink/datastream/state_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ class StateBackend(object, metaclass=ABCMeta):

The :class:`EmbeddedRocksDBStateBackend` stores working state in an embedded
`RocksDB <http://rocksdb.org/>`_, instance and is able to scale working state to many
terrabytes in size, only limited by available disk space across all task managers.
terabytes in size, only limited by available disk space across all task managers.

**Raw Bytes Storage and Backends**

Expand Down Expand Up @@ -244,7 +244,7 @@ def get_db_storage_paths(self) -> List[str]:
Gets the configured local DB storage paths, or null, if none were configured.

Under these directories on the TaskManager, RocksDB stores its SST files and
metadata files. These directories do not need to be persistent, they can be ephermeral,
metadata files. These directories do not need to be persistent, they can be ephemeral,
meaning that they are lost on a machine failure, because state in RocksDB is persisted
in checkpoints.

Expand Down
2 changes: 1 addition & 1 deletion flink-python/pyflink/table/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -1405,7 +1405,7 @@ def deserialize(self, datum):
# We should be careful here. The size of these types in python depends on C
# implementation. We need to make sure that this conversion does not lose any
# precision. Also, JVM only support signed types, when converting unsigned types,
# keep in mind that it requires 1 more bit when stored as singed types.
# keep in mind that it requires 1 more bit when stored as signed types.
#
# Reference for C integer size, see:
# ISO/IEC 9899:201x specification, chapter 5.2.4.2.1 Sizes of integer types <limits.h>.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,7 @@ private boolean isWindowLate(W window) {

/**
* Returns the cleanup time for a window, which is {@code window.maxTimestamp +
* allowedLateness}. In case this leads to a value greated than {@link Long#MAX_VALUE} then a
* allowedLateness}. In case this leads to a value greater than {@link Long#MAX_VALUE} then a
* cleanup time of {@link Long#MAX_VALUE} is returned.
*
* @param window the window whose cleanup time we are computing.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -492,7 +492,7 @@ public static Config getConfig(
* @param externalAddress optional tuple of external address and port to be reachable at. If
* null is given, then a Pekko config for local actor system will be returned
* @param bindAddress optional tuple of bind address and port to be used locally. If null is
* given, wildcard IP address and the external port wil be used. Takes effect only if
* given, wildcard IP address and the external port will be used. Takes effect only if
* externalAddress is not null.
* @param executorConfig config defining the used executor by the default dispatcher
* @return Pekko config
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ public static <T extends KeyedStateHandle> int findTheBestStateHandleForInitial(
return bestHandlePos;
}

/** Helper class tha defines a key-range in RocksDB as byte arrays for min and max key. */
/** Helper class that defines a key-range in RocksDB as byte arrays for min and max key. */
private static final class KeyRange {
static final KeyRange EMPTY = KeyRange.of(new byte[0], new byte[0]);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -585,7 +585,7 @@ public void dispose() {
if (db != null) {

// Metric collection occurs on a background thread. When this method returns
// it is guaranteed that thr ForSt reference has been invalidated
// it is guaranteed that the ForSt reference has been invalidated
// and no more metric collection will be attempted against the database.
if (nativeMetricMonitor != null) {
nativeMetricMonitor.close();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -206,8 +206,8 @@ public ForStDBPutRequest<K, N, UV> buildDBPutRequest(StateRequest<?, ?, ?, ?> st
}

/**
* Build a request for bunch put. Maily used for {@link StateRequestType#MAP_PUT_ALL} and {@link
* StateRequestType#CLEAR}.
* Build a request for bunch put. Mainly used for {@link StateRequestType#MAP_PUT_ALL} and
* {@link StateRequestType#CLEAR}.
*
* @param stateRequest The state request.
* @return The {@code ForStDBBunchPutRequest}.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -723,7 +723,7 @@ public void setLocalDbStoragePaths(String... paths) {
* Gets the configured local DB storage paths, or null, if none were configured.
*
* <p>Under these directories on the TaskManager, ForSt stores some metadata files. These
* directories do not need to be persistent, they can be ephermeral, meaning that they are lost
* directories do not need to be persistent, they can be ephemeral, meaning that they are lost
* on a machine failure, because state in ForSt is persisted in checkpoints.
*
* <p>If nothing is configured, these directories default to the TaskManager's local temporary
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -691,7 +691,7 @@ public void mergeStateHandlesWithClipAndIngest(
notImportableHandles);

if (exportedColumnFamilyMetaData.isEmpty()) {
// Nothing coule be exported, so we fall back to
// Nothing could be exported, so we fall back to
// #mergeStateHandlesWithCopyFromTemporaryInstance
int bestStateHandleForInit =
findTheBestStateHandleForInitial(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -557,7 +557,7 @@ public void dispose() {
IOUtils.closeQuietly(writeBatchWrapper);

// Metric collection occurs on a background thread. When this method returns
// it is guaranteed that thr ForSt reference has been invalidated
// it is guaranteed that the ForSt reference has been invalidated
// and no more metric collection will be attempted against the database.
if (nativeMetricMonitor != null) {
nativeMetricMonitor.close();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -690,7 +690,7 @@ private long doPhysicalRemoveAndGetValue(long node, long prevNode, long nextNode
}

/**
* Physically delte the meta of the node, including the node level index, the node key, and
* Physically delete the meta of the node, including the node level index, the node key, and
* reduce the total size of the skip list.
*
* @param node node to remove.
Expand Down
Loading